The MCP caching contract

What the specification now provides. tools/list, prompts/list, resources/list, resources/read, and resources/templates/list return a CacheableResult with two required fields. ttlMs is a freshness hint in milliseconds. cacheScope is "public" or "private" and controls whether a shared intermediary may cache the response. These complement listChanged notifications rather than replacing them. Separately, servers SHOULD return tools in a deterministic order explicitly to improve LLM prompt cache hit rates.

Why ordering is a cost question. Provider prompt caches match on exact prefixes. One reordered tool definition changes a byte in the preamble and invalidates everything below it. Cached input reads cost roughly a tenth of fresh input across all three major providers; Anthropic additionally charges 1.25x to write a five-minute cache and 2.0x for an hour. So a nondeterministically-ordered tool list does not merely miss the cache — it pays the write multiplier repeatedly, which is worse than not caching at all.

At 30,000 tokens of tool definitions and a million calls a day, the gap between a stable and an unstable preamble is roughly an order of magnitude on the largest single line of the bill. This is the highest-leverage item on this list and it is a sorting function.

Architecture.

1. 01Two distinct caches. A protocol cache (client-side, honouring ttlMs and cacheScope) and the provider prompt cache (server-side, keyed on prefix bytes). They are unrelated mechanisms and both matter. Do not conflate them.
2. 02Respect `cacheScope` strictly. "private" must not enter a shared tier. In a multi-tenant gateway this is a data-isolation control, not a performance hint. Key private entries by principal.
3. 03Canonicalize before hashing. Sort tools by a stable key, canonicalize JSON — key order, whitespace, number formatting — then serialize. Never trust the server's ordering even though it SHOULD be deterministic; normalize on receipt.
4. 04Layer the preamble by volatility. Most static first: system prompt, then core tool definitions, then retrieved tools, then conversation. Cache breakpoints go at the boundaries. Anything that changes per request belongs below every breakpoint.
5. 05Treat `ttlMs` as a hint, `listChanged` as truth. Subscribe via subscriptions/listen for toolsListChanged where the server supports it, and fall back to TTL polling where it does not. TTL alone gives you a stale window equal to the TTL; the notification closes it.
6. 06Instrument the hit rate as a first-class metric. Cache hit ratio on the prompt cache belongs on the same dashboard as latency and error rate. A prompt change that drops it from 90% to 20% is a ten-fold cost regression that no functional test will catch.

Stack. LiteLLM to normalize cache headers and token accounting across providers. Langfuse to track hit rate per prompt version, which is what makes a regression attributable. Anthropic's count_tokens endpoint for billing-accurate measurement including tool definitions — it is free, and estimating tool-definition cost with a tokenizer library will be wrong.

Where answers fail. Describing an HTTP cache and stopping. The question is whether you know that provider prompt caching and protocol caching are different systems, and that a sort order is worth a large fraction of the inference bill.