Skip to main content

The Context Bloat Problem and Mitigations

The more MCP servers you add, the more LLM context their tool definitions and results consume. This guide organizes that problem, drawing on the discussion in "MCP is dead. Long live the CLI" and related articles, and explains mitigations on both the server and the client side.

1. What This Document Covers​

  • What consumes context
  • Measuring the size of the problem
  • The claim in MCP is dead, and later updates
  • Server-side mitigations
  • Client-side mitigations (progressive discovery, dynamic server connection, programmatic tool calling)
  • Interaction with caching, choosing a threshold, and the habit of measuring

What consumes context​

MCP crowds the context through two paths.

  1. Always-loaded tool definitions
    • Many clients hand the model the definitions (name / description / inputSchema) of every tool on every connected server at the start of a conversation. Definitions of tools that are never used are included every time
  2. Intermediate data in tool results
    • When the model calls a tool directly, the result always passes through the model's context. For "attach the Google Drive meeting transcript to the Salesforce lead", the full transcript passes through the model twice: once when fetched and once when written. Large data can exceed the context limit, and copying mistakes become more likely

Both come from naive client implementations that load everything upfront and pass every intermediate result through the model, not from the MCP specification itself.

Measuring the size of the problem​

Quandri's engineering blog extracted the tool definitions (name, description, and parameter JSON schema) from MCP servers connected in a real environment and measured them.

MCP serverToolsEstimated tokens
Linear42about 12,807
Notion14about 4,039
Slack12about 3,792
Postgres9about 438
Total77about 21,077

Definitions alone for the four servers take about 10.5% of a 200K context and about 16.5% of a 128K context. Linear alone is about 12,800 tokens, so all 42 definitions stay resident even if you only ever use get_issue and save_issue.

These figures use a rough estimate of about 4 characters per token, and the full-server numbers are extrapolated from sampled averages. Re-measure in your own environment by actually connecting.

The claim in MCP is dead, and later updates​

On context, Eric Holmes's "MCP is dead. Long live the CLI" (2026-02-28) argues:

  • LLMs already know CLIs from man pages and shell scripts on GitHub, so no special protocol or resident definitions are needed
  • A CLI can filter results through pipes, jq, and grep. With MCP, you either put huge results straight into the context or build filtering into the server

For example, you can count the changes in a Terraform plan without putting intermediate data into the context.

terraform show -json plan.out | jq '[.resource_changes[] | select(.change.actions[0] == "no-op" | not)] | length'

On the other hand, Quandri's post adds that after the measurements, Claude Code introduced Tool Search (deferred loading), which loads MCP tool definitions on demand and reduces context usage by 85% or more. Current Claude Code loads only tool names and server instructions at session start and fetches definitions when needed. In other words, definition bloat is a problem of naive implementations that preload everything, and client-side work can reduce it substantially.

Two issues remain. Intermediate data in tool results is not solved unless the server and client are designed for it. And the fact that CLIs and Skills can be lighter in some situations still holds (see MCP vs CLI vs Skills - Comparison and When to Use Each for details).

Server-side mitigations​

What server authors can do:

  • Reduce the number of tools
    • Do not create one tool per API endpoint. Group by what the user is trying to do
  • Keep descriptions short and precise
    • Say briefly what the tool does and when to use it. Claude Code truncates each tool description and each server's instructions at 2,048 characters by default, so put critical details first
  • Provide filter and pagination arguments
    • Offer search criteria, field selection, and limit and cursor so results do not grow larger than needed
  • Define outputSchema
    • Declaring the structure of results lets clients generate typed APIs for Code Mode, and lets the model handle only the fields it needs
  • Return tools in a deterministic order
    • In 2026-07-28, list results have a deterministic order, which keeps upstream prompt caches stable across reconnects

Design details are covered in Tool Design Patterns and Anti-patterns.

Client-side mitigation 1: progressive discovery​

Progressive discovery controls when tool definitions enter the context.

  • The host fetches definitions with tools/list as usual but does not inject them into the model's context
  • Instead, it gives the model a lightweight meta-tool such as search_tools
  • Only the definitions that are needed are loaded into the context

In the MCP docs' figure, loading everything upfront costs about 150,000 tokens, while loading only what the task requires costs about 2,000.

One implementation pattern is a three-layer structure.

LayerRole
Catalogsearch_tools takes a natural-language query and returns only tool names and one-line descriptions
InspectFetches the full definition, including the input schema, for a candidate tool
ExecuteCalls the tool with full knowledge of its interface

Choose a search strategy from the following.

StrategyCharacteristics
KeywordBM25 or regex. Simple, and effective when tool names and descriptions are clear
EmbeddingVector similarity over descriptions. Better with synonyms and semantic closeness
SubagentA small, fast model picks the tools. Accurate, but relatively costly
HybridCombine rankings, or switch strategies by query

OpenAI and Anthropic provide tool search on the platform side. When it is available, you can prefer it over a custom implementation. Build your own when the provider does not offer one or when you need domain-specific ranking or access-control filtering.

When implementing, keep these points in mind.

  • Offer multiple detail levels (name only, name and description, full schema)
  • Memoize fetched definitions host-side so re-injecting them does not need another tools/list
  • Re-index the search catalog when notifications/tools/list_changed arrives
  • Present tools grouped by their source server

Client-side mitigation 2: dynamic server connection​

Progressive discovery also extends from individual tools to entire servers. Instead of connecting to every server at startup:

  1. Keep a registry of available servers with short descriptions
  2. Connect to a server only when the model decides it needs that server
  3. Disconnect servers that are no longer needed and free the context

This suits general-purpose agents, where the user's intent is not known upfront. Keep always-on servers to a minimum and connect the rest as needed. Combined with Skills, a skill can declare the MCP servers it needs, and the host connects them only when that skill is invoked.

Client-side mitigation 3: programmatic tool calling (Code Mode)​

Programmatic tool calling changes how tools are invoked. Instead of calling tools directly, the model writes code that calls them, and the code runs in a sandbox. Chaining several tools, filtering results, and deduplication happen inside the sandbox, and only the final result returns to the model.

In the MCP docs' figure, 100,000+ tokens of intermediate results passing through the model become a script of about 200 tokens and a summary of about 15 tokens. Anthropic's engineering blog likewise reports that presenting tools as a code API reduced token usage from 150,000 to 2,000, a 98.7% saving. Cloudflare calls a similar approach "Code Mode".

It works in three steps.

  1. The host generates a typed API usable inside the sandbox from the MCP tool schemas (return types are precise when outputSchema is present)
  2. The model writes a script that calls that API
  3. The sandbox runs the script, and the host brokers function calls and relays them to MCP servers. The model sees only console.log output or the final return value

Code Mode brings in a code execution surface, so safety design is mandatory.

  • Give the sandbox no network access, and route all external communication through the host broker
  • Keep authorization tokens and credentials in the host, never in generated code
  • Approving a script does not approve every tool call it makes at runtime. The broker evaluates each call against the grant
  • Treat a tool result from one server as untrusted input to another
  • Set timeouts and memory limits, and validate and truncate sandbox output before returning it to the model

Tool errors arrive as a successful response with isError: true, so have generated wrappers convert them into exceptions so model-written code can use try / catch.

Interaction with caching​

MCP caching and prompt caching are separate, and you need to consider both.

  • MCP caching
    • Results such as tools/list, server/discover, and resources/read carry ttlMs (milliseconds the result may be considered fresh) and cacheScope (public or private). Clients follow them, and treat a cached list as stale as soon as a list_changed notification arrives, even before the TTL expires. Do not use the TTL as a polling interval; check freshness when the data is needed and re-fetch only if stale
  • Prompt caching
    • Most providers cache the prompt prefix, including the tools array. Adding or removing tool definitions mid-conversation invalidates that cache, and the miss can cost more tokens than the definitions you removed

To preserve prompt caching:

  • Append newly discovered definitions after the cache breakpoint rather than re-sorting the tools array
  • Route every call through a single stable call_tool({name, args}) meta-tool so the array never changes
  • Treat server disconnection as a conversation-boundary operation rather than a per-turn one

Choosing a threshold​

While there are few tool definitions, loading all of them is fine. As a rule of thumb, switch to progressive discovery once definitions exceed 1 to 5% of the context. The MCP client best practices also recommend a threshold expressed as a percentage of the context window. In Claude Code, ENABLE_TOOL_SEARCH=auto:5 sets a 5% threshold, and when unset, all MCP tools are deferred.

The habit of measuring​

Decide by measuring, not by feeling that something looks heavy.

  • Before connecting, count the tokens of the tools/list result
  • With the same prompt, vary progressive discovery and the search strategy, and check that the correct tool is actually selected
  • Check result sizes. Claude Code warns when an MCP tool's output exceeds 10,000 tokens and caps output at 25,000 tokens by default (an oversized result is saved to a file and the conversation gets its path)

2. Comparison of Mitigations​

MitigationWhat it controlsTrade-off
Fewer tools and shorter descriptionsDefinition sizeDesign effort. Cutting too much removes capability
Progressive discovery (Tool Search)When definitions enter the contextDepends on search accuracy. Watch prompt cache invalidation
Dynamic server connectionNumber of connected serversConnection and authorization latency
Code ModeIntermediate results passing through the contextRequires building a sandbox and ensuring its safety
Filter arguments and paginationSize of tool resultsMore complex tool design
Skills + CLILoads only needed instructions on demandAssumes a CLI exists and the agent has a shell

3. Common Pitfalls​

Progressive discovery is on, but the right tool is not chosen​

If search accuracy is low, the model cannot find the tool it needs. Put the words users actually use (verbs and objects) into tool names and descriptions. Writing in the server instructions what kinds of tasks the server handles and when to search, as Claude Code's Tool Search encourages, also helps. For the few tools needed on every turn, use a setting such as alwaysLoad that keeps them from being deferred.

Changing the tools array breaks the prompt cache​

If you add or remove tool definitions mid-conversation through progressive discovery or dynamic connection, the prompt cache is invalidated, and the miss can cost more tokens than you thought you saved. Limit changes to conversation boundaries or additions after the cache breakpoint.

Ignoring a stale list_changed​

Even if you cache the tool list, treat it as stale and re-fetch when list_changed arrives. Notifications are best effort and can be lost, so also use ttlMs-based refreshing.

A tool with no filter arguments returns a huge result​

A tool with neither pagination nor filtering can return tens of thousands of tokens in one call. Even if the client truncates it, the information you needed is lost. Provide limit, field selection, and search criteria on the server.

Overtrusting the Code Mode sandbox​

Running generated code requires isolation. If you do not satisfy all of: no credentials in the code, no network access, and an approval policy applied per call, you add paths for prompt injection. Do not skip the safety design in favor of saving context.

Assuming the CLI solves everything​

A CLI keeps no definitions resident and can filter through pipes, but it assumes the agent has a shell, a mature CLI exists, and the environment is already authenticated. Where those assumptions do not hold, use MCP together with progressive discovery or Code Mode.

Connecting every server without measuring​

Each added server adds definitions and can lower the model's selection accuracy. Do not connect servers you do not use; enable them only when needed.

4. Code Examples​

Measuring the size of tool definitions​

Save the tools/list result and estimate from the character count (the same rough 4 characters per token as Quandri). Confirm the actual token count with the tokenizer of the model you use.

# tools-list.json holds the full tools/list response
chars=$(jq -c '.result.tools[]' tools-list.json | wc -c)
tools=$(jq '.result.tools | length' tools-list.json)
echo "tools=${tools} chars=${chars} approx_tokens=$((chars / 4))"

# Share of a 200K context (%)
echo "scale=1; ${chars} / 4 * 100 / 200000" | bc

The three layers of progressive discovery​

An example from the MCP docs. The model starts with a lightweight search and fetches the full definition only for the tool it needs.

// Layer 1: Catalog. Return only names and one-line descriptions
search_tools({ query: "update salesforce record" });
// → [
//     { name: "salesforce_updateRecord", description: "Update fields on a Salesforce object" },
//     { name: "salesforce_upsertRecord", description: "Insert or update based on external ID" }
//   ]

// Layer 2: Inspect. Fetch the full definition for the candidate only
get_tool_details({ name: "salesforce_updateRecord" });

// Layer 3: Execute. Call it with full knowledge of the interface
salesforce_updateRecord({ objectType: "Lead", recordId: "00Q...", data: { Status: "Working" } });

Code Mode: filtering in the sandbox​

A script written by the model calls typed functions that the host generated from outputSchema. Thousands of log entries are deduplicated inside the sandbox, and only a one-line summary returns to the model.

// Model-generated code that runs in the sandbox
const logs = await logging_getLogs({
  level: "error",
  since: Date.now() - 3600000,
});

// Filter and deduplicate in the sandbox, not in the model's context
const uniqueErrors = new Map<string, LogEntry>();
for (const log of logs.entries) {
  if (!uniqueErrors.has(log.message)) uniqueErrors.set(log.message, log);
}

for (const [message, log] of uniqueErrors) {
  await ticketing_createIssue({
    title: `Error: ${message}`,
    body: `First seen: ${log.timestamp}`,
    priority: "high",
  });
}

// Only this single line returns to the model
console.log(`Filed ${uniqueErrors.size} tickets from ${logs.entries.length} error logs`);

Server side: filter arguments and outputSchema​

Give the caller control over result size with limit / cursor / fields, and declare the structure with outputSchema.

{
  "name": "issues_search",
  "description": "Search issues by status and assignee. Returns at most `limit` items.",
  "inputSchema": {
    "type": "object",
    "properties": {
      "status": { "type": "string", "enum": ["open", "closed"] },
      "assignee": { "type": "string" },
      "fields": {
        "type": "array",
        "items": { "type": "string", "enum": ["id", "title", "status", "url"] },
        "description": "Fields to include. Defaults to id and title."
      },
      "limit": { "type": "integer", "minimum": 1, "maximum": 50, "default": 20 },
      "cursor": { "type": "string" }
    }
  },
  "outputSchema": {
    "type": "object",
    "properties": {
      "items": { "type": "array", "items": { "type": "object" } },
      "nextCursor": { "type": "string" }
    },
    "required": ["items"]
  }
}

A call_tool meta-tool to preserve prompt caching​

A setup that funnels execution through one stable meta-tool so the tools array does not change when tools are added or removed.

{
  "name": "call_tool",
  "description": "Call a tool found with search_tools.",
  "inputSchema": {
    "type": "object",
    "properties": {
      "name": { "type": "string" },
      "args": { "type": "object" }
    },
    "required": ["name"]
  }
}

Configuration in Claude Code​

# Switch to deferred loading once definitions exceed 5% of the context
ENABLE_TOOL_SEARCH=auto:5 claude

# Turn off deferral and preload all tools
ENABLE_TOOL_SEARCH=false claude

# Change the output token limit for tools (default is 25,000)
MAX_MCP_OUTPUT_TOKENS=50000 claude

For a server you use on every turn, you can exempt it from deferral in .mcp.json.

{
  "mcpServers": {
    "core": {
      "type": "http",
      "url": "https://example.com/mcp",
      "alwaysLoad": true
    }
  }
}

5. Summary​

  • Two things consume context: resident tool definitions, and intermediate data in tool results that passes through the model
  • Quandri measured about 21,000 tokens (about 10.5% of 200K) for definitions alone across 4 servers and 77 tools
  • Definition bloat is a problem of naive implementations that preload everything, and deferred loading such as Tool Search reduces it substantially
  • On the server side, trim tools, keep descriptions short, and provide filter arguments and outputSchema
  • On the client side, choose among progressive discovery, dynamic server connection, and Code Mode depending on whether you want to reduce definitions or results
  • Changing the tools array mid-conversation breaks the prompt cache. Limit changes to conversation boundaries or additions after the cache breakpoint
  • Switch to progressive discovery once definitions exceed 1 to 5%, and confirm the effect by measuring before and after

References​