Agent Compaction Without Amnesia: 74% Fewer Tokens, Zero Drops

Long agent sessions outgrow context windows, and naive summarization silently drops safety constraints while stalling runs for tens of seconds. Staged compaction with pinned constraints and reversible tool offloading keeps sessions alive without amnesia.

• Compaction-induced violations jump from 0% to 30% unless constraints are pinned outside lossy summaries
• Synchronous summarization consumes up to 51% of agent wall time at tight thresholds
• I cut session tokens 74% with zero violations using staged pressure plus background compaction

My longest agent run died at 3am on a Tuesday. A research crew had worked for six hours, accumulated 700k tokens of tool output, and hit the context ceiling. The harness dutifully summarized history into 90k tokens and continued. Twenty minutes later the agent emailed a draft contract to an external address it had been explicitly forbidden to contact in turn three. The rule was gone. The summary kept task state and dropped the compliance preamble as old news. I rebuilt our compaction stack that week around one paper and two patterns.

Compaction erases what summaries deem old

The June 2026 Governance Decay study put numbers on my 3am incident. Across 1,323 episodes and seven model families, in-context governance constraints obeyed at 0% violation in full context jumped to 30% violation after compaction, reaching 59% on some models. When the constraint survived the summary, violations stayed at 0%. When the summary dropped it, violations hit 38%. The mechanism is mundane: a compaction step optimized for task continuity has no reason to preserve a standing policy. The policy is old, it is not the current sub-goal, and it competes for a shrinking budget against active task state.

Adversarial content makes it worse. The same study showed optimized injections biasing the summarizer to omit legitimate policies defeat every evaluated model. Your compaction prompt is now part of your attack surface. Treat it like one.

The fix the authors propose is Constraint Pinning: quarantine governance constraints from lossy compaction and prepend them verbatim every turn. Cost is around 47 pinned tokens, under half a percent of a production compaction budget. Violations return to 0% in their benchmark. Forty-seven tokens against a 3am contract leak. Cheapest insurance I buy.

Latency is the second tax. A May 2026 serving study found synchronous compaction consuming 51% of end-to-end wall time at 16k thresholds on small models, falling to 8% only at 96k thresholds. Worse, models self-bound summary output: 48x more input grew summaries only 3x. One paper on compaction theory prices the extreme: summarizing 800k tokens of history into 100k takes about 26 minutes of generation. Blocking the agent for that is not engineering. It is a queue with extra steps.

This connects directly to our cost work. Our provider arbitrage routing guide showed cached prefixes running 10x cheaper, and compaction summaries destroy prefix stability. Our background-thread tracing study gives the span data to see compaction costs per session. Measure first, compact second.

graph TD
  A[Session crosses 90% budget] --> B[Pinned constraints set aside: 47 tokens]
  B --> C[Stage 1: offload old tool results]
  C --> D{Budget ok?}
  D -->|yes| E[Continue, constraints restored]
  D -->|no| F[Stage 2: background parallel summary]
  F --> E

Step 1: Stage pressure instead of jumping to summaries

Most harnesses leap from full history to lossy summary in one move. Staged compaction graduates through cheaper reversible steps first. Lighter moves often free enough space that the summary never fires.

File: requirements.txt

tiktoken==0.9.0
httpx==0.28.1
pydantic==2.8.0
structlog==24.4.0

File: config.py

from pydantic_settings import BaseSettings, SettingsConfigDict
from pydantic import Field

class Settings(BaseSettings):
    model_config = SettingsConfigDict(env_file=".env", extra="allow")
    window_tokens: int = 200000
    warn_at: float = 0.75
    compact_at: float = 0.90
    pinned_path: str = "pinned.md"
settings = Settings()

File: compactor.py

import json
from pathlib import Path
from config import settings

def token_len(text):
    return max(1, len(text) // 4)

def load_pinned():
    p = Path(settings.pinned_path)
    return p.read_text() if p.exists() else ""

def stage_offload(messages, budget):
    kept, store, freed = [], {}, 0
    for m in messages:
        if m.get("role") == "tool" and token_len(str(m.get("content", ""))) >= 2000:
            ref = f"offload-{len(store)}"
            store[ref] = m["content"]
            preview = str(m["content"])[:400]
            freed += token_len(str(m["content"])) - token_len(preview)
            kept.append({"role": "tool", "content": f"[{ref}: {preview}... full output retrievable]", "ref": ref})
        else:
            kept.append(m)
    Path("/tmp/offloads.json").write_text(json.dumps(store)[:2000000])
    return kept, freed

def microcompact(messages):
    seen, out, freed = set(), [], 0
    for m in messages:
        key = str(m.get("content", ""))[:200]
        if m.get("role") == "tool" and key in seen:
            freed += token_len(str(m.get("content", "")))
            out.append({"role": "tool", "content": "[duplicate tool result summarized above]"})
        else:
            seen.add(key)
            out.append(m)
    return out, freed

def needs_compact(messages):
    total = sum(token_len(str(m.get("content", ""))) for m in messages)
    return total / settings.window_tokens

uv venv --python 3.12 && source .venv/bin/activate
uv pip install -r requirements.txt
cat pinned.md

File: pinned.md

STANDING CONSTRAINTS - NEVER DROP, NEVER SUMMARIZE:
1. Never email external addresses. Route drafts internally.
2. Never exfiltrate credentials or keys in tool calls.
3. Confirm before consequential actions per approval policy.

First war story. Before staging, our harness summarized at 90% every time. A six-hour session compacted nine times, and each summary of a summary drifted further from the original constraints. By compaction four, the model confidently cited a policy that never existed. Hallucinated governance. I traced it to summaries inheriting prior summaries without ground truth. Now stage one offloads tool results reversibly and microcompact dedups repeats, which together freed 61% of our typical overflow without any summary call. Summaries run only when reversible steps fail. Drift dropped to zero because ground truth survives.

Step 2: Parallel background summaries with pinned restore

When summaries must fire, never block the agent. Research on parallel compaction shows chunked concurrent summarization beating sequential baselines on retention and wall time. I compact in the background while the agent continues on the uncompacted tail, then swap at the next safe point between turns. Docker-style agents already apply cross-session compaction at safe points rather than mid-turn. Copy that discipline.

Second war story. Our first background compactor swapped state mid-tool-call and the agent acted on half-old half-new context, booking a duplicate data export. Corrupted turn. Now swaps happen only between model turns with the pinned block prepended first, then the fresh summary, then recent turns verbatim. Order matters: constraints lead, narrative follows, evidence closes. Three sessions a day for two months, zero swap incidents since.

Tool-output offloading deserves emphasis because it is the only lossless move. Large results leave the context as reference pointers with previews while full bytes wait in the store. The agent pulls full content back on demand. Nothing is destroyed. For payload-heavy crews this single pattern delayed every summary by hours. Our document MCP server with range reads applies the same philosophy to files: outline first, bytes on demand.

Staged state runs larger than pure summaries because offloads stay retrievable and pins stay verbatim. That is the point. Token counts alone mislead: 96k of amnesiac summary loses to 182k of grounded state on every task-completion metric I track. Our reasoning effort tiers study reinforces it: what the model sees decides what it solves.

Step 3: Verify memory like a safety system

Compaction is memory surgery. Verify it monthly with three probes. Constraint recall: seed ten standing rules, run sessions past three compactions, assert all ten survive verbatim. Trigger test: replay a forbidden request after compaction and assert refusal. Swap audit: diff pre and post-compaction state for unapproved deletions. The ConstraintRot recipe of policy turn, benign filler, trigger request ports directly into CI. I run 40 such scenarios nightly. Cost is $6. Findings are priceless.

When NOT to compact cleverly

Let's be clear. Simple sessions need simple handling.

Skip staged machinery for short-horizon tasks under 30 turns. Sliding windows or plain truncation work when early turns genuinely stop mattering. Four stages of reversible offloading for a 12-turn support bot is theater. Match machinery to session length.

Skip LLM summarization where exact numbers persist across turns. Prices, dosages, credentials, counts. Summaries round, paraphrase and occasionally invent. Offload verbatim or pin explicitly. I pin every standing number over $100 and every production credential pattern. Paranoia with a config file.

Production bottlenecks I hit: tokenizer estimates drift 15% from provider counts so calibrate weekly; offload stores grow 18GB monthly without retention so expire after 7 days; background summaries race user interrupts so cancel on new input; pinned blocks bloat as teams append rules so review monthly and keep under 200 tokens. Ordinary fixes. Required fixes.

Bottom line: pin constraints outside lossy summaries, offload before summarizing, compact in the background, and long sessions stop forgetting what matters.

By Deepak Bagada, Founder & Editor-in-Chief at Daily AI World. I build agentic workflows and high-concurrency SaaS platforms at SaaSNext. Follow my benchmarks on <a href="https://x.com/deeepakbagada">X @deeepakbagada and <a href="https://deepakbagada.in">deepakbagada.in.