Tool namespace explosion

The constraint. These are two problems with one cause and different solutions. The token cost is fixed by caching (item 10). The selection accuracy is not — a model choosing among 400 near-synonymous tools is doing retrieval, badly, with no reranker.

Architecture.

1. 01Retrieve tools, do not enumerate them. Index tool descriptions. At request time, retrieve the 15-30 relevant tools and expose only those. This is ordinary retrieval and it responds to ordinary retrieval technique — hybrid search, reranking, a threshold.
2. 02Tier by frequency. A small always-present core of high-frequency tools, plus a retrieved tail. The core stays in the cached prefix; the tail varies. This preserves most of the cache benefit while keeping selection tractable.
3. 03Namespace and deduplicate. Forty servers will have four search tools and three get_user. Prefix by server and, more importantly, resolve the genuine duplicates — the accuracy loss is largely the model picking a plausible wrong one among identical-sounding options.
4. 04Treat descriptions as the prompt they are. Tool descriptions and parameter metadata are model-facing text that is rarely reviewed. Enforce a house style: what it does, when to use it, when not to use it, and one example. The negative case is what disambiguates siblings.
5. 05Progressive disclosure for large surfaces. A single list_capabilities tool that returns a compact menu, followed by a call that expands one area, beats 400 definitions in the preamble. Costs a round trip, buys the window back.

Measure selection, not schema conformance. Grammar-constrained decoding has made JSON validity a non-issue; it tells you nothing about whether the right tool was chosen with the right arguments. Build a labelled set of query-to-correct-tool pairs and score selection directly. Sixty percent is only actionable once you know whether it is retrieval failure, description ambiguity, or argument construction.

Security note that belongs in the same review. Tool descriptions are an injection surface — tool poisoning, where instructions hidden in descriptions or parameter metadata reach the model but never the user, is an OWASP-catalogued class, with cross-tool poisoning and rug-pull server updates as variants. Forty servers is forty supply chains. Review tool metadata as code, pin by digest, and gate write-capable tools behind approval.

Stack. Any vector store for the tool index — this is a tiny corpus, so pgvector or an in-process index is sufficient. FlashRank for CPU-only reranking at negligible latency. A private registry for pinning. promptfoo for the selection eval matrix.

Where answers fail. Fixing the token cost with caching and declaring victory. Caching a 60,000-token preamble makes the wrong answer cheap.