AI agents are burning context and budget: how to compress tool output by 99% without losing meaning
I’m Denis Shokhirev, an Agentic AI Systems Architect in Freiburg im Breisgau, Germany, and at DennisCraft AI Studio I ship production AI systems on Claude, Supabase, n8n, Doppler, and self-hosted Postgres for DACH B2B clients. The failure mode I see most often in live agent systems is not reasoning quality but agents dumping raw tool output into context: 30,000 lines of logs, oversized JSON payloads, or full query results that burn tokens, latency, and budget in one step. This is a design probl
I’m Denis Shokhirev, an Agentic AI Systems Architect in Freiburg im Breisgau, Germany, and at DennisCraft AI Studio I ship production AI systems on Claude, Supabase, n8n, Doppler, and self-hosted Postgres for DACH B2B clients. The failure mode I see most often in live agent systems is not reasoning quality but agents dumping raw tool output into context: 30,000 lines of logs, oversized JSON payloads, or full query results that burn tokens, latency, and budget in one step.
This is a design problem, not a model problem. Most teams tune prompts, switch models, and argue about RAG, but they still let every tool speak in paragraphs when the agent only needs a handful of facts. Once you fix that, cost drops fast and answer quality usually improves because the model sees less noise.
Why tool output is the hidden cost center
Agent pipelines are usually built around tools: SQL, code scanners, web retrieval, workflow engines, and internal APIs. Each tool returns data in its own format and at its own verbosity level. If you pass that raw output straight into the model, you force the model to do parsing, ranking, filtering, and summarization inside the context window. That is the most expensive place to do it.
Anthropic’s tool use documentation recommends structured tool inputs and outputs rather than dumping unbounded text into the model context. Source: Anthropic Docs, Tool Use, 2024, https://docs.anthropic.com/ . OpenAI’s Cookbook examples on function calling and long-context handling show the same operational principle: select relevant information first, then send it to the model. Source: OpenAI Cookbook, 2024, https://cookbook.openai.com/ .
What the model actually needs
| Tool output source | Raw payload | What the agent needs | Compression method |
|---|---|---|---|
| Postgres query | 1000+ rows | aggregate, outliers, top records | SQL aggregation and field projection |
| semgrep / bandit | large JSON report | severity, file, line, rule, short message | schema normalization |
| n8n workflow logs | verbose execution trace | status, failing node, error message | event filtering |
| retrieved docs | whole pages | relevant paragraph and citation | chunk reranking and deduplication |
That distinction matters. An agent almost never needs the entire payload. It needs a decision object. Once you start designing tool interfaces around that idea, 99% compression is normal, not exceptional.
Compression should happen before the model
The biggest mistake is asking the model to summarize raw tool output after you already paid to place it into context. If the tool produces noise, remove it before the model sees it. In practice I use three layers: filter, normalize, aggregate.
1. Filter by decision relevance
If a code review agent only needs critical findings, don’t send informational warnings. Keep severity, file path, line number, rule identifier, and a short evidence string. Store the full report elsewhere for audit.
def compact_bandit(report: dict) -> dict:
findings = []
for item in report.get("results", []):
if item.get("issue_severity") not in {"HIGH", "MEDIUM"}:
continue
findings.append({
"file": item.get("filename"),
"line": item.get("line_number"),
"severity": item.get("issue_severity"),
"test_id": item.get("test_id"),
"message": item.get("issue_text", "")[:160]
})
return {
"count": len(findings),
"findings": findings[:20]
}This turns a huge scanner payload into something an agent can actually reason over. You preserve the signal and drop the ceremony.
2. Normalize schema across tools
One tool returns message, another returns description, a third uses nested metadata. If the agent keeps re-learning each structure, you are wasting context. Map all tool results into a common schema like source, severity, object, evidence, and recommended_action.
3. Aggregate instead of forwarding lists
Agents often ask business questions that should be answered in SQL, not by shipping table dumps into a model. If the task is “summarize payment failures in the last 24 hours,” then the database should produce the summary.
psql "$DATABASE_URL" -c "
SELECT
status,
count(*) AS total,
max(updated_at) AS last_seen
FROM payments
WHERE updated_at >= now() - interval '24 hours'
GROUP BY status
ORDER BY total DESC;
"That one query often replaces thousands of tokens. The meaning is preserved because the business question was aggregated by nature.
Keep auditability without polluting context
A common objection in regulated environments is traceability. Teams worry that once output is compressed, they lose the raw evidence needed for incident review. The fix is simple: separate operational context from forensic storage.
The agent gets a compact summary object plus a reference to raw data. The full payload goes into self-hosted Postgres, Supabase storage, or a structured event log. This keeps the prompt small while preserving reconstruction paths.
type ToolSummary = {
tool: string;
summary: object;
rawRef: string;
checksum: string;
};
export function makeToolSummary(
tool: string,
summary: object,
rawRef: string,
checksum: string
): ToolSummary {
return { tool, summary, rawRef, checksum };
}This pattern also helps with data minimization. OWASP’s Top 10 for LLM Applications warns against unnecessary exposure of sensitive data to models and external systems. Source: OWASP GenAI Security Project, 2025, https://genai.owasp.org/ . Sending only the minimum required facts reduces both cost and risk.
Where to place compression in a real stack
Do not hide this logic in prompts. Put it where the output is produced: SQL queries, Python post-processing, an n8n function node, or an adapter around the Anthropic SDK. Compression is infrastructure, not prose.
steps:
- tool: postgres
query: "select status, count(*) as total from jobs group by status"
- transform:
keep:
- status
- total
sort_by: total
limit: 10
- model:
input: compact_resultThe rule I use is blunt. If tool output is more than 10x larger than the question the agent is trying to answer, it probably needs compression. If it is 100x larger, that is no longer an optimization opportunity; it is a system design bug.
On my recent deployments, the biggest savings did not come from moving to a cheaper model. They came from refusing to let tools talk too much. When the agent receives a compact, typed, decision-ready object, it behaves more consistently and costs less per task.
FAQ
Do I always need an LLM to compress tool output?
No. Deterministic methods are usually better: SQL aggregation, JSON filtering, schema mapping, and severity thresholds. Use an LLM only when you need semantic merging from unstructured sources.
Will compression hurt reasoning quality?
Usually the opposite. If you compress to the actual task, you remove distracting tokens and improve focus. The model performs worse on noise than on concise evidence.
How do I measure whether this works?
Track tokens per tool call, latency per completed task, and task success rate on a fixed evaluation set. Those three metrics show the effect quickly.
Where should I keep full raw output?
Store it in Postgres, Supabase storage, or a dedicated event log, and pass a reference into the agent summary object. Raw output should be retrievable, not automatically injected into prompts.
Does this matter only for very large models?
No. Smaller and cheaper models benefit even more because they degrade faster under noisy context and have tighter cost targets.
In your production agent stack, which stage is currently doing the most damage: oversized SQL results, workflow logs, static analysis payloads, or retrieval context that no one trimmed before inference? I’d genuinely like to know.
I run a free 30-min stack audit for DACH founders building AI in regulated markets. DM me on LinkedIn or write to @ger_dennis_ai.
I build agents like this for businesses in Germany and the EU.
Tell me in two sentences what eats your team’s time – reply within 24 hours.
Turn your process into an AI system
Production quality. DACH B2B focus.