Dynamic Tool Discovery and the Limits of Finding the Right Tool — Why Retrieval Doesn't Fix a Raw MCP Surface

In the 200-tool bottleneck post we argued that handing an agent a raw catalog of every endpoint your systems expose is slow, expensive, and unreliable. The industry agrees — and has an answer: dynamic tool discovery. Don’t inject 200 schemas on every turn. Let the agent search for the tool it needs.

It’s a real technique and you should use it. But two things are quietly assumed: that search reliably finds the right tool, and that finding it is the expensive part. Neither holds. Search changes how an agent finds a tool. It changes nothing about what that tool does once found — and that’s where most of the cost, most of the latency, and all of the governance live.

What dynamic tool discovery actually is

Several converging techniques share one umbrella:

• Progressive disclosure. Tool names are advertised cheaply; full schemas are fetched only for candidates — practical thanks to MCP’s tool-list pagination and listChanged.
• Search-as-a-tool. One meta-tool — “find me a tool that does X” — backed by retrieval over the catalog. Anthropic’s Tool Search Tool is the best-known implementation.
• Server-side orientation. MCP servers expose a catalog resource or ship prompts explaining how to use their tools, so guidance travels with the server.

The shared premise — the catalog is large and mostly irrelevant on any given turn, so pay for it lazily — is correct. The per-turn schema tax is the most visible cost in a naive MCP setup, and discovery removes a good chunk of it. Credit where it’s due.

First question: does search actually find the right tool?

Every tool-search demo assumes retrieval lands on the correct tool, first try. Over 200 raw endpoints it frequently doesn’t, for structural reasons rather than implementation bugs.

A tool catalog is a hostile corpus. Retrieval works when documents are semantically distinct and the distinguishing detail is in the text being embedded. Tool catalogs violate both:

• Near-duplicates are the norm. getOrder, getOrderById, getOrderDetails, lookupOrderV2, searchOrders are maximally similar in embedding space and functionally different — and it worsens as the catalog grows, which is the regime you reached for search because of.
• The deciding detail usually isn’t in the description. A required scope, a pagination behavior, a deprecation status, a side effect. Two tools can have near-identical prose and completely different blast radius. Most descriptions are auto-generated from OpenAPI summary fields written to fill a docs page, not to serve as a retrieval key.
• Semantic proximity isn’t functional safety. “Cancel the order” and “delete the order” are close neighbours in embedding space and far apart in consequence. Ranking has no notion of which mistakes are cheap — or of which tools are deprecated, or callable by this credential.

And failure is silent. In document RAG, a bad hit is an obviously irrelevant passage a human notices. In tool retrieval, a bad match is a plausible tool that runs. Pick getOrderSummary instead of getOrderDetails and nothing errors — you get a 200 with data in it, and the agent proceeds with the wrong shape. Retrieval is normally deployed with a human reading the results; here the consumer is a probabilistic system with no verification step, acting on the output.

Accuracy compounds, and real tasks are multi-hop

A single lookup tolerates a decent hit rate. A four-system task needs four correct retrievals in a row, and accuracy multiplies:

Even excellent per-hop retrieval yields a task-level failure rate nobody would ship — and 95% is what you get on a small, clean catalog, which is precisely the thing you didn’t have. A capability collapses those hops into one declared flow: no retrieval step to get wrong, so nothing to compound.

The trade nobody states: tokens for turns

Nobody retrieves top-1. You retrieve top-k — five, ten — so you load k schemas, not one. Every miss costs a re-search plus a retry. Effective cost is k × (1 + retry_rate) schemas.

The token saving survives; 200 schemas down to ten is still a large cut. But the turn count goes up:

Dynamic discovery trades tokens for turns.

For batch and chat, usually a good trade. For a real-time voice assistant — the 200-tool post case — it can be net negative: a search step is a round-trip that didn’t exist before, and search turns and retry turns are dead air. You optimized what you were measuring and worsened what the caller experiences.

What it fixes, and what it doesn’t

Discovery is a partial fix for the inbound metadata problem and silent on selection reliability, outbound payloads, turn count, and trust. Take the three biggest misses in turn.

1. Orchestration doesn’t shrink — it just gets found faster

“Where’s my order? It was supposed to arrive yesterday.” needs the CRM to identify the caller, the order system for their latest order, shipping for live status, and the scheduler if it’s late. With discovery, the agent searches, calls, carries the ID across, searches again… four systems, four sequential round-trips, plus the search turns that didn’t exist before. Each tool got cheaper to load; the conversation got longer. On a phone call, that’s audible.

A capability collapses it into one call, because the sequencing is declared rather than improvised:

  aggregates:
    - display: Order Status
      namespace: support
      flows:
        where-is-my-order:
          description: Identify caller, find latest order, get live shipping status.
          inputParameters:
            phone: { type: string, required: true }
          steps:
            identify:
              type: call
              call: crm.find-by-phone
              with: { phone: support.phone }
            latest-order:
              type: call
              call: orders.latest-for-customer
              with: { customerId: $.identify.id }
            track:
              type: call
              call: shipping.get-status
              with: { trackingId: $.latest-order[0].trackingId }
          outputParameters:
            # A flat, speakable shape — the model doesn't re-derive it mid-call.
            orderId: { type: string,  mapping: $.latest-order[0].id }
            status:  { type: string,  mapping: $.track.status }
            etaText: { type: string,  mapping: $.track.humanEta }
            isLate:  { type: boolean, mapping: $.track.status == "delayed" }

Four upstream hops, zero model turns. No amount of retrieval quality produces that, because retrieval’s job ends the moment a tool is selected.

2. Response shaping is untouched — and it’s the bigger token sink

Here’s the asymmetry. Tool schemas are paid once per turn and are what everyone measures. Tool responses are paid on every call, are frequently far larger, and nobody optimizes them.

A raw GET /orders/{id} returns line items, addresses, tax breakdown, fulfillment history, audit stamps, and a pagination envelope. The agent needed four fields. The rest enters context anyway, sits there for the conversation, and is re-processed every turn. Discovery has no opinion about this — it found you the tool that returns the bloat. The outputParameters block above declares four fields, because four is what the job needs: an order-of-magnitude cut discovery never sees.

3. Governance isn’t a context-window problem

Discovery tells the agent a tool exists. It says nothing about whether the agent may call it, under which identity, with which credential scope, against which environment, with what audit trail. Those are answered at the capability boundary — where identity, policy, credential binding, and invocation meet. We covered that in the capability lifecycle: mocks, a YAML review gate, synthetic staging, then least-privilege keys bound at a second gate. Retrieval sits entirely outside it. It’s an optimization, not a control.

The gap nobody talks about: how does the agent know what to search for?

Search needs a query. At the moment it needs a tool, the agent knows its goal — “file this expense” — but not the vocabulary of your surface — POST /v3.1/reimbursement_requests. That’s vocabulary mismatch, and better retrieval doesn’t fix it: the agent can’t query for a word it has no reason to know.

Only three places can bridge goal-language to tool-vocabulary:

1. Out-of-band, client-side. An Agent Skill primes the agent on which families exist and what words to search with. Mature and effective — but hand-maintained, in a second artifact that drifts.
2. In-band, server-side. The server ships orientation itself, as a catalog resource or prompt templates. Where the protocol is genuinely evolving, and the better long-term home.
3. In the tool surface itself. Name and shape tools in the language of the task, not the API. An agent seeking “check the order” against a catalog containing order-status needs no translation. It needed one because the catalog said getOrderByIdV2.

Option 3 is the one people skip, and the only one that removes work rather than relocating it. Which is the point of the whole post:

Retrieval doesn’t remove curation. It relocates it.

So: is discovery a threat to properly crafted capabilities?

No — it’s the best consumer of them. Discovery is retrieval over a corpus, and its quality is bounded by that corpus. A capability is what makes the corpus good: task-shaped names, intent-aligned descriptions, deterministic behavior, shaped responses, one governed surface.

Point excellent retrieval at raw API sprawl and you get a faster path to a tool that still costs five turns and a 4,000-token payload. Point mediocre retrieval at a curated catalog and it lands first try, because there are only a handful and they’re named after what the agent is trying to do. And because the exposes block that defines the MCP tool also projects the REST endpoint and the Agent Skill, the orientation that primes discovery can’t drift from the tool discovery finds.

What this means in practice

Use dynamic tool discovery — it’s a real win on the metadata axis and there’s no reason to fight it. Don’t mistake it for a strategy: turning on tool search partly addresses the schema tax and leaves selection accuracy, turn count, payload size, retry rate, and governance exactly where they were.

Curate the surface anyway. Roughly ranked by impact:

Discovery owns the bottom row, and even there it’s a trade. It is not the top four.

So judge a tool surface by two questions discovery can’t answer: does the agent pick correctly on the first try, and once it does, how many turns, how many tokens, and how much trust does the call cost? Both are set when the tool is authored. No retrieval layer changes either afterwards.

The takeaway

Dynamic tool discovery solves an agent drowning in tool descriptions. It doesn’t solve an agent picking the wrong one out of two hundred near-identical candidates — silently, and multiplicatively across every hop. And it does nothing about the tool work: the orchestration improvised, the payloads waded through, the retries paid for twice, the credentials nobody wanted it to hold.

Search narrows the list. Shape decides both the odds and the cost. A properly crafted capability is what turns a found tool into a cheap, deterministic, governed one — and what makes the search land on it in the first place.

• 🔊 Voice assistants and the 200-tool bottleneck
• 🔐 The capability lifecycle — from mocked APIs to trusted production
• 🧩 Author capabilities in your editor with Crafter 1.0 beta2
• 📖 Documentation
• 🛝 Playground