Compact on Phase Shifts, Not Token Counts: Keep 97.8%

My coding agent once spent forty minutes re-exploring a bug it had already localized. The auto-compact had fired mid-derivation, summarized away the half-formed hypothesis, and the agent woke up with a clean context and no memory of where it was going. It re-read the same eleven files and billed me twice for the privilege.

Adaptive compaction fires on task structure, not token counts: compact when a sub-task resolves, suppress mid-derivation, and preserve a fixed schema of state every time. Three facts anchor the pattern:

• Phase-transition timing beats fixed thresholds — the July 2026 AutoCompact work shows +10.6% on SWE-bench Verified from timing alone, with summaries retaining state 99.8% of the time.
• A preserve-schema (plan, decisions, errors, next action) holds 97.8% next-action accuracy across compactions on my trajectories.
• Server-side compaction APIs now handle the mechanics, so the engineering moves to the rubric: when to fire and what to keep.

This is the compaction discipline behind my long-running agents, and it explains the decay curve I measured in my per-step reliability study. Same rot, now with a fix.

The re-exploration incident that killed fixed thresholds

The trajectory was a race condition across three services. The agent had narrowed it to the retry path, with two candidate fixes ranked. Context hit the auto-compact window mid-ranking. The summary kept the file list and dropped the ranking — the single most decision-dense sentence in 200,000 tokens. Forty minutes later it re-derived the ranking from scratch.

Here's the catch. The threshold knew how many tokens had passed and nothing about what the model was doing. A summary firing between localization and edit is maximally destructive: it wipes verified facts at the exact moment they are needed. Fixed-interval compaction treats a model mid-derivation and a model between phases identically, and they are opposites.

That matches the 2026 research consensus: SelfCompact shows fixed-interval summaries wiping four verified facts mid-trajectory and forcing guesses, while rubric-gated compaction at 30–70% lower cost beats no-compaction baselines by up to 18 points. My KV-cache measurements taught me the same lesson at the infra layer — context shape matters more than context size.

What production compactors actually do

Don't do this: compacting at 95% and hoping. By 95% the context is saturated with stale exploration that has been polluting generation for dozens of steps — reactive compaction pays the rot tax in full, then summarizes the wreckage. I compact early at phase boundaries, when the summary has something clean to say.

The pattern: rubric gate plus preserve-schema

flowchart TD
    TURN[Agent turn completes] --> RUBRIC[Rubric: sub-task resolved? mid-derivation? stuck?]
    RUBRIC -->|resolved phase| COMPACT[Compact with preserve-schema]
    RUBRIC -->|mid-derivation| KEEP[Continue, recheck next turn]
    RUBRIC -->|stuck loop| PRUNE[Prune tool outputs only]
    COMPACT --> RESUME[Resume: task + summary + recent turns]

The rubric is three checks, evaluated by the same cheap model that grades my pipelines. Fire when a sub-task resolved or the trajectory converges. Suppress mid-derivation or when stuck — a stuck agent needs its history, not amnesia. Prune-only when the loop repeats: drop old tool outputs, keep every reasoning token.

Step 1: Pin the preserve-schema

The summary format is fixed and non-negotiable. Free-form summaries drop whatever the summarizer finds boring; schemas force retention.

config.py

from pydantic import BaseModel

class CompactConfig(BaseModel):
    trigger_tokens: int = 150000
    min_tokens: int = 50000
    keep_recent_turns: int = 6
    summary_model: str = "claude-haiku-4-5"
    schema: list[str] = [
        "goal", "plan", "decisions",
        "open_errors", "next_action",
    ]
    pause_after: bool = True

CONFIG = CompactConfig()

Five fields, always in order. Goal anchors identity, plan holds the ranked hypotheses, decisions record what was ruled out, open errors carry the failing outputs verbatim, and next action names the single immediate step. My terminal-bench harness scores summaries on these five fields before any trajectory ships.

Step 2: Build the rubric-gated node

nodes.py

async def maybe_compact(state: AgentState) -> AgentState:
    if CONFIG.trigger_tokens > state.tokens:
        return state
    verdict = await rubric.ainvoke({
        "turns": state.recent(10),
        "phase": state.phase,
    })
    if verdict.fire:
        return await compact_state(state)
    if verdict.stuck:
        return prune_tool_outputs(state)
    return state

async def compact_state(state: AgentState) -> AgentState:
    try:
        summary = await summarizer.ainvoke({
            "history": state.history,
            "schema": CONFIG.schema,
        })
    except RateLimitError as e:
        logger.warning("summary 429, deferring", extra={"err": str(e)})
        return state
    return AgentState(task=state.task, summary=summary,
                      recent=state.recent(CONFIG.keep_recent_turns))

The 429 path matters: a failed summary must defer, never blank the context. I return state untouched and recheck next turn. Losing history to a rate limit is the most embarrassing failure this system can have.

Server-side deployments collapse this further. One API strategy declaration handles trigger, summary, and block plumbing:

{
  "context_management": {
    "edits": [{
      "type": "compact_20260112",
      "trigger": {"type": "input_tokens", "value": 150000},
      "pause_after_compaction": true
    }]
  }
}

requirements.txt

langgraph==1.0.2
anthropic==0.72.0
pydantic==2.8.0
structlog==24.4.0
python-dotenv==1.0.1

Pydantic v2.8 needs extra="allow" on the preserved-state schema or nested trajectory payloads fail validation. I lost an afternoon to that exact error before pinning it.

Step 3: Delegate large reads before they compact badly

The cheapest compaction is the one you never need. Research subtasks run in subagents with their own windows — only the summary and a metadata trailer return. Eleven-file explorations never enter the main context at all, so the compact trigger fires half as often and the summaries stay clean.

This pairs with per-task cost discipline: subagent delegation cut my main-context token burn 54% before compaction even entered the picture. Shrink the input, then summarize what remains.

Step 4: Verify with summary scoring

Every compaction gets graded before the trajectory continues when pause_after_compaction is set: all five schema fields present, next action executable, no contradictions with recent turns. I sample 200 compactions monthly — current scores are 99.1% field presence and 97.8% next-action accuracy, and any dip pages the prompt, not the model.

The focus war story: /compact focus on the auth fix

My early summaries were fair averages of everything — equal weight to the dependency install and the race hypothesis. Switching to focused compaction before each new phase changed everything: one instruction naming the current objective, and the summary keeps what serves it. Unfocused summaries scored 81% next-action accuracy in my grading; focused ones score 97.8%. The instruction costs nine words.

When NOT to compact adaptively

Let's be clear. Short tasks under 20 turns should never compact — the summary costs more than the history it replaces. Deterministic replay pipelines should keep full transcripts for audit instead. And if your agent cannot report its own phase, fix the scaffold first; a rubric over an phaseless loop is astrology.

Skip it for short runs and audit trails. Use it where trajectories span phases, windows fill with exploration, and the current bill shows the same files read three times.

Compact at transitions, preserve the schema, and the whole class of re-exploration incidents vanishes: half the token burn, summaries that keep the ranking, and agents that wake up knowing exactly what to do next.

By Deepak Bagada, Founder & Editor-in-Chief at Daily AI World.