Dynamic Tool Discovery and the Limits of Finding the Right Tool — Why Retrieval Doesn't Fix a Raw MCP Surface

In the 200-tool bottleneck post we argued that handing an agent a raw catalog of every endpoint your systems expose is slow, expensive, and unreliable. The industry agrees — and has an answer: dynamic tool discovery. Don’t inject 200 schemas on every turn. Let the agent search for the tool it needs.

It’s a real technique and you should use it. But two things are quietly assumed: that search reliably finds the right tool, and that finding it is the expensive part. Neither holds — and the vendors shipping it say so in their own documentation. Search changes how an agent finds a tool. It changes nothing about what that tool does once found — and that’s where most of the cost, most of the latency, and all of the governance live.

What dynamic tool discovery actually is

Several converging techniques share one umbrella:

• Progressive disclosure. Tool names are advertised cheaply; full schemas are fetched only for candidates — practical thanks to MCP’s tool-list pagination and listChanged.
• Search-as-a-tool. One meta-tool — “find me a tool that does X” — backed by retrieval over the catalog. Anthropic’s Tool Search Tool is the best-known implementation.
• Server-side orientation. MCP servers expose a catalog resource or ship prompts explaining how to use their tools, so guidance travels with the server.

The shared premise — the catalog is large and mostly irrelevant on any given turn, so pay for it lazily — is correct. The per-turn schema tax is the most visible cost in a naive MCP setup, and discovery removes a good chunk of it. Credit where it’s due.

First question: does search actually find the right tool?

Every tool-search demo assumes retrieval lands on the correct tool, first try. Over 200 raw endpoints it frequently doesn’t, for structural reasons rather than implementation bugs.

A tool catalog is a hostile corpus. Retrieval works when documents are semantically distinct and the distinguishing detail is in the text being embedded. Tool catalogs violate both:

• Near-duplicates are the norm. getOrder, getOrderById, getOrderDetails, lookupOrderV2, searchOrders are maximally similar in embedding space and functionally different — and it worsens as the catalog grows, which is the regime you reached for search because of.
• The deciding detail usually isn’t in the description. A required scope, a pagination behavior, a deprecation status, a side effect. Two tools can have near-identical prose and completely different blast radius. Most descriptions are auto-generated from OpenAPI summary fields written to fill a docs page, not to serve as a retrieval key.
• Semantic proximity isn’t functional safety. “Cancel the order” and “delete the order” are close neighbours in embedding space and far apart in consequence. Ranking has no notion of which mistakes are cheap — or of which tools are deprecated, or callable by this credential.

This isn’t a fringe complaint. Anthropic’s own tool-search documentation states that Claude’s “ability to pick the right tool degrades once you exceed 30–50 available tools,” and Cloudflare’s Code Mode post puts it bluntly: “If you present an LLM with too many tools, or overly complex tools, it may struggle to choose the right one or to use it correctly.”

And failure is silent. In document RAG, a bad hit is an obviously irrelevant passage a human notices. In tool retrieval, a bad match is a plausible tool that runs. Pick getOrderSummary instead of getOrderDetails and nothing errors — you get a 200 with data in it, and the agent proceeds with the wrong shape. Retrieval is normally deployed with a human reading the results; here the consumer is a probabilistic system with no verification step, acting on the output.

Accuracy compounds, and real tasks are multi-hop

A single lookup tolerates a decent hit rate. A four-system task needs four correct retrievals in a row, and accuracy multiplies (illustrative arithmetic, not measured):

Even excellent per-hop retrieval yields a task-level failure rate nobody would ship — and 95% is what you get on a small, clean catalog, which is precisely the thing you didn’t have.

Anthropic counters that tool search keeps selection accuracy high “even across thousands of tools,” because it loads only a focused set. Notice what that concedes: accuracy is a function of how few relevant tools sit in context. Search achieves that at retrieval time, by guessing from a query. Curation achieves it at design time, by construction. Same mechanism — the difference is whether you’re betting on a query or relying on a design. And a capability collapses the four hops into one declared flow, so there’s no retrieval step to get wrong and nothing to compound.

The trade nobody states: tokens for turns

Nobody retrieves top-1 — Anthropic’s tool search returns up to five matching tools per search by default. So you load k schemas, not one, and every miss costs a re-search plus a retry. Effective cost is k × (1 + retry_rate) schemas.

The token saving is real: Anthropic reports a typical multi-server setup (GitHub, Slack, Sentry, Grafana, Splunk) burning ~55k tokens in definitions, and tool search cutting that by over 85%. Worth having. But the turn count goes up:

Dynamic discovery trades tokens for turns.

For batch and chat, usually a good trade. For a real-time voice assistant — the 200-tool post case — it can be net negative: a search step is a round-trip that didn’t exist before, and search turns and retry turns are dead air. You optimized what you were measuring and worsened what the caller experiences.

What it fixes, and what it doesn’t

Discovery is a partial fix for the inbound metadata problem and silent on selection reliability, outbound payloads, turn count, and trust. Take the three biggest misses in turn.

1. Orchestration doesn’t shrink — it just gets found faster

“Where’s my order? It was supposed to arrive yesterday.” needs the CRM to identify the caller, the order system for their latest order, shipping for live status, and the scheduler if it’s late. With discovery, the agent searches, calls, carries the ID across, searches again… four systems, four sequential round-trips, plus the search turns that didn’t exist before. Each tool got cheaper to load; the conversation got longer. On a phone call, that’s audible.

A capability collapses it into one call, because the sequencing is declared rather than improvised:

  aggregates:
    - display: Order Status
      namespace: support
      flows:
        where-is-my-order:
          description: Identify caller, find latest order, get live shipping status.
          inputParameters:
            phone: { type: string, required: true }
          steps:
            identify:
              type: call
              call: crm.find-by-phone
              with: { phone: support.phone }
            latest-order:
              type: call
              call: orders.latest-for-customer
              with: { customerId: $.identify.id }
            track:
              type: call
              call: shipping.get-status
              with: { trackingId: $.latest-order[0].trackingId }
          outputParameters:
            # A flat, speakable shape — the model doesn't re-derive it mid-call.
            orderId: { type: string,  mapping: $.latest-order[0].id }
            status:  { type: string,  mapping: $.track.status }
            etaText: { type: string,  mapping: $.track.humanEta }
            isLate:  { type: boolean, mapping: $.track.status == "delayed" }

Four upstream hops, zero model turns. No amount of retrieval quality produces that, because retrieval’s job ends the moment a tool is selected.

This is also just good tool design, independent of Naftiko. Anthropic’s guidance on writing tools for agents is explicit: instead of list_users, list_events and create_event, build schedule_event; instead of get_customer_by_id, list_transactions and list_notes, build get_customer_context. Tools should “offload agentic computation from the agent’s context back into the tool calls themselves.” A capability is that principle made declarative.

2. Response shaping is untouched — and it’s the bigger token sink

Here’s the asymmetry. Tool schemas are paid once per turn and are what everyone measures. Tool responses are paid on every call, are frequently far larger, and nobody optimizes them.

A raw GET /orders/{id} returns line items, addresses, tax breakdown, fulfillment history, audit stamps, and a pagination envelope. The agent needed four fields. The rest enters context anyway, sits there for the conversation, and is re-processed every turn. Discovery has no opinion about this — it found you the tool that returns the bloat. The outputParameters block above declares four fields, because four is what the job needs: an order-of-magnitude cut discovery never sees. Anthropic measures the same effect in their Slack tools — a “concise” response format uses about a third of the tokens of a “detailed” one.

3. Governance isn’t a context-window problem

Discovery tells the agent a tool exists. It says nothing about whether the agent may call it, under which identity, with which credential scope, against which environment, with what audit trail. Those are answered at the capability boundary — where identity, policy, credential binding, and invocation meet. We covered that in the capability lifecycle: mocks, a YAML review gate, synthetic staging, then least-privilege keys bound at a second gate. Retrieval sits entirely outside it. It’s an optimization, not a control.

The strongest counter-argument: let the agent write code

The most serious answer to everything above isn’t tool search — it’s code execution. Cloudflare’s Code Mode converts MCP tools into a TypeScript API and has the model write code against it. Anthropic’s code execution with MCP applies the same idea, reporting one workflow dropping from 150,000 tokens to 2,000. Both address what search doesn’t: orchestration runs in a loop the model writes, and results are filtered in the sandbox before reaching context.

The insight is correct, and it’s the same one underneath a capability: the work belongs outside the context window. Where the approaches diverge is which artifact does the work.

Code execution puts improvised, per-run code in a sandbox. That buys generality at costs Anthropic names directly — “running agent-generated code requires a secure execution environment with appropriate sandboxing, resource limits, and monitoring… operational overhead and security considerations that direct tool calls avoid.” Add the ones a regulated buyer cares about: the code is regenerated each run, so behavior isn’t reproducible; there’s no artifact to review before it touches a golden source; and cost varies per invocation.

A capability puts the same logic in a declarative spec. Same escape from the context window, different trust properties: deterministic, versioned, reviewable in a pull request, bound to least-privilege credentials at a gate.

And the two converge. Anthropic’s post closes by recommending agents persist their working code as reusable functions with a SKILL.md, building “a toolbox of higher-level capabilities.” That’s the destination. A capability is what you get when you skip the improvisation and author the artifact directly.

The gap nobody talks about: how does the agent know what to search for?

Search needs a query. At the moment it needs a tool, the agent knows its goal — “file this expense” — but not the vocabulary of your surface — POST /v3.1/reimbursement_requests. That’s vocabulary mismatch, and better retrieval doesn’t fix it: the agent can’t query for a word it has no reason to know.

Only three places can bridge goal-language to tool-vocabulary:

1. Out-of-band, client-side. An Agent Skill primes the agent on which families exist and what words to search with. Mature and effective — but hand-maintained, in a second artifact that drifts.
2. In-band, server-side. The server ships orientation itself, as a catalog resource or prompt templates. Where the protocol is genuinely evolving, and the better long-term home.
3. In the tool surface itself. Name and shape tools in the language of the task, not the API. An agent seeking “check the order” against a catalog containing order-status needs no translation. It needed one because the catalog said getOrderByIdV2.

Option 3 is the one people skip, and the only one that removes work rather than relocating it. Tellingly, Anthropic’s own optimization tips for tool search recommend options 1 and 3 by hand: “add a system prompt section describing available tool categories,” and “use keywords in descriptions that match how users describe tasks.” Both are curation, performed by a human, out of band. The feature’s manual tells you to do the work the feature was supposed to remove. Which is the point of the whole post:

Retrieval doesn’t remove curation. It relocates it.

So: is discovery a threat to properly crafted capabilities?

No — it’s the best consumer of them. Discovery is retrieval over a corpus, and its quality is bounded by that corpus. A capability is what makes the corpus good: task-shaped names, intent-aligned descriptions, deterministic behavior, shaped responses, one governed surface.

Point excellent retrieval at raw API sprawl and you get a faster path to a tool that still costs five turns and a 4,000-token payload. Point mediocre retrieval at a curated catalog and it lands first try, because there are only a handful and they’re named after what the agent is trying to do. And because the exposes block that defines the MCP tool also projects the REST endpoint and the Agent Skill, the orientation that primes discovery can’t drift from the tool discovery finds.

What this means in practice

Use dynamic tool discovery — it’s a real win on the metadata axis and there’s no reason to fight it. Don’t mistake it for a strategy: turning on tool search partly addresses the schema tax and leaves selection accuracy, turn count, payload size, retry rate, and governance exactly where they were.

Curate the surface anyway. Roughly ranked by impact:

Discovery owns the bottom row, and even there it’s a trade. It is not the top four.

So judge a tool surface by two questions discovery can’t answer: does the agent pick correctly on the first try, and once it does, how many turns, how many tokens, and how much trust does the call cost? Both are set when the tool is authored. No retrieval layer changes either afterwards.

The takeaway

Dynamic tool discovery solves an agent drowning in tool descriptions. It doesn’t solve an agent picking the wrong one out of two hundred near-identical candidates — silently, and multiplicatively across every hop. And it does nothing about the tool work: the orchestration improvised, the payloads waded through, the retries paid for twice, the credentials nobody wanted it to hold.

Search narrows the list. Shape decides both the odds and the cost. A properly crafted capability is what turns a found tool into a cheap, deterministic, governed one — and what makes the search land on it in the first place.

Further reading

• 🧠 Code Mode: the better way to use MCP — Cloudflare, on why LLMs call tools worse than they write code
• 🧠 Code execution with MCP — Anthropic, on tool definitions and intermediate results as the two token sinks
• 🧠 Writing effective tools for agents — Anthropic, on consolidating tools and shaping responses

• 🧠 Tool search tool — Anthropic docs, including the 30–50 tool accuracy cliff

• 🔊 Voice assistants and the 200-tool bottleneck
• 🔐 The capability lifecycle — from mocked APIs to trusted production
• 🧩 Author capabilities in your editor with Crafter 1.0 beta2
• 📖 Documentation
• 🛝 Playground