Progressive tool discovery and agent observability at scale

An operations agent connected to five MCP servers loads 60 tool definitions at the start of every session: tens of thousands of tokens before the user types anything. Traces show that almost every session uses three to five tools, nearly always the same ones, and that the agent increasingly picks a wrong tool with a similar name. Next quarter two more servers are being added.

The cost of monolithic context

Loading everything upfront (monolithic context) has two costs that grow with every tool:

• Tokens: the definitions are paid for on every request, used or not.
• Selection: with dozens of similar tools, the accuracy of picking the right one drops. That’s the more serious symptom, because cheaper tokens don’t fix it.

If tools are unnecessary or overlap, they’re removed or consolidated first (lesson 1). This lesson is for when all of them are legitimate, but each session uses few.

Progressive discovery

With progressive discovery, the agent starts with little and brings in what it needs when it needs it. In the Claude API, tool search implements it like this:

1. You send all the definitions, but mark with defer_loading the ones that shouldn’t enter the context upfront.
2. At first the model sees the search tool and the non-deferred tools.
3. When it needs something, it searches the catalog (names, descriptions and parameters) and the API loads only the matches.
4. Always keep loaded the few tools used in almost every session, so they don’t need to be searched for.

With MCP servers connected through the API’s connector, deferral is configured per server or per tool. And since search uses the descriptions, they must be clear and use the words a user would: cutting them to one line worsens selection.

Option in the scenario	What it solves
Caching the definitions	Lowers token cost; choosing among 60 stays the same
One-line descriptions	Fewer tokens and worse selection
One subagent per tool	Multi-agent by default: more cost and coordination
Tool search with the most-used tools loaded	Tokens, selection and scale when more servers arrive

Observability: seeing every step

A booking agent handles 50,000 sessions a day, each with several calls to the model and to availability, pricing and payment systems. There are intermittent failures (wrong prices quoted, half-finished bookings) and only the final answer is stored. Rerunning the input almost always works. With that, there’s no way to know which step failed.

What’s needed is per-session tracing:

• A span for every model call and every tool call.
• In each span: inputs, outputs, tokens, latency, stop_reason and the prompt and model version.
• A session identifier that ties all the steps together.

At this scale, storing everything in full is expensive. The solution is sampling: a fraction of successful sessions and all failing ones. If the company already has a tracing backend for its microservices, reuse it. And since traces contain customer data, the same retention and masking rules as the rest of the logs apply.

The usual traps

• Caching as the solution to a tool selection problem.
• Trimming descriptions to save tokens.
• Also deferring the tools used in almost every session.
• Dumping every request’s full prompt to standard output, unstructured.
• Asking customers to describe what happened.
• Manually reproducing intermittent failures that don’t appear when the input is rerun.

In the exam

If the scenario describes many tools loaded, few used per session and wrong choices, look for tool search with the most-used tools always loaded; a mention of “more servers next quarter” reinforces that answer. If it describes a multi-step agent with failures that can’t be diagnosed, look for traces with a span per call, sampled successes and all failures kept. Discard options that only make things cheaper or that depend on someone’s account of events.