SDSignal Desk

Gemini's API stops stuffing every tool into the prompt

· Updated · Signal Desk newsroom

Analysis of reporting by Google AI Blog. How we work

Image: Google AI Blog

Tool Retrieval lets agents keep large tool libraries on standby and load schemas only when needed. It is an unglamorous fix for one of the real scaling problems in agent design, with cost savings that depend on your traffic.

Key takeaways

  • Tool Retrieval is in preview in the Gemini Interactions API and works with Gemini 3.6 and later, including all Gemini 4 models.
  • Developers add a tool_search tool and mark tools with defer_loading; the model sees names and descriptions but loads full schemas only when needed.
  • Search, schema loading and the function call happen inside one request, and each step appears in the interaction log for debugging.
  • A client-side mode lets teams use their own registry or permission filters, at the cost of an extra round trip.
  • There is no separate charge, but loaded schemas are billed as input for the rest of the conversation, so savings vary.

The problem with a hundred tools

Agents get more useful as they connect to more systems, and then they get worse. Every tool's full parameter schema is sent with every request, which inflates token counts, adds latency and, as Google itself notes, lowers tool-selection accuracy as the candidate list grows. Anyone who has wired an agent to a large internal API or a busy MCP server has hit this wall.

Google's answer, Tool Retrieval, is now in preview in the Gemini API's Interactions API. Developers add a tool_search tool to a request and set defer_loading on the tools to hold back. The model still sees each deferred tool's name and description, so it knows what exists, but the parameter schemas stay out of context until the model asks for one by name.

How it works in practice

Google's example is a customer-service request to refund a damaged order. Within a single call, the model searches for the refund tool, the API loads its schema and the model emits the function call. All three steps appear in the interaction record, which matters for debugging: when an agent picks the wrong tool, teams can see what it considered. Google says there is no index to build or host, since the search runs server-side, and recommends keeping frequently used tools loaded while deferring the long tail.

Two details stand out. First, it works with remote MCP servers: set defer_loading on an MCP server and every tool it exposes loads on demand. That makes large servers practical without listing tools one by one. Second, it is designed around prefix caching, because deferred schemas are added later in the conversation and the cached prefix stays stable from turn to turn.

For teams with their own registry, embedding index or per-user permission rules, a client-side mode hands the search to the app. That costs one extra round trip per search, but it allows exposing new tools mid-conversation, for example when a user connects an integration. For regulated environments, keeping permission checks in your own code is probably the more defensible choice.

The cost math is honest, and conditional

Google does not overclaim on savings. Tool Retrieval has no separate fee; turning it on adds a small fixed block of input tokens, and each schema the model loads is billed as input for the rest of the conversation, just like a tool declared up front. You save when the schemas you never load outweigh those you do, which Google calls the usual case for large libraries, and it tells developers to measure against their own traffic. That is the right framing.

The bigger risk is retrieval quality. If an agent depends on names and descriptions to find tools, sloppy descriptions now cost accuracy in a new way, and a missed tool may produce a confident wrong path rather than an error. This feature rewards teams that treat tool metadata as carefully as code. It builds on the API direction we tracked with Gemini 4 Argon's developer release.

Who feels it

Agent developers
Large tool libraries become cheaper and faster to expose, if descriptions are good.
MCP server builders
Deferred loading makes big servers practical for Gemini-based agents.
Enterprises
Client-side retrieval keeps permission filtering in-house while trimming context.
Rival API providers
Tool search is quickly becoming a baseline expectation for agent platforms.

What to watch

  1. Published accuracy and latency comparisons between deferred and fully loaded tool sets.
  2. When Tool Retrieval leaves preview and whether it reaches APIs beyond Interactions.
  3. Developer reports on missed or wrong tool retrievals in production.
  4. Similar features from competing model APIs.

FAQ

What is Tool Retrieval in the Gemini API?

A preview feature in the Interactions API that lets agents defer loading tool parameter schemas until the model requests a specific tool, reducing tokens and latency for large tool libraries.

Does Tool Retrieval cost extra?

No separate charge. It adds a small fixed block of input tokens, and loaded schemas are billed as input for the rest of the conversation.

Which models support Tool Retrieval?

Gemini 3.6 and later, including all Gemini 4 models, in the Interactions API.

Does it work with MCP servers?

Yes. Setting defer_loading on a remote MCP server tool loads each of its tools on demand.

Read the original

Continue at the source.

Google AI Blog

Companies: Google

More on Google

All Google coverage →