Context Compaction

Replacing older conversation or agent history with a model-written summary as the window fills: client vs server-side implementations (Anthropic, OpenAI), tool-result clearing, context resets, failure modes and how to choose.

Contents

Context compaction is the technique of replacing the older part of a conversation or agent trajectory with a summary, written by a model, once the history approaches a token limit, so that the run can continue in a bounded context window. The summary takes the place of the turns it covers. The model then continues from “summary + recent turns + new input” instead of the full transcript (Anthropic compaction, Anthropic context engineering). It is lossy by design. It belongs to a family of history-reduction techniques alongside rule-based tool-result clearing (drop old observations without summarising them), context resets (start a fresh agent from a hand-written handoff) and external memory (write state to files the agent re-reads). Compaction matters even when the window is not full, because quality degrades as context grows (Chroma). It is one of the main levers of context engineering and harness engineering.

Why compact before the hard limit

  • Overflow. A long agent run grows by every tool call and result. Manus reports about 50 tool calls per typical task and roughly 100:1 input-to-output tokens (Manus).
  • Context rot. Recall falls as input length grows, on every model tested, even for trivial tasks (Chroma). Anthropic frames context as a finite “attention budget” (Anthropic), and its compaction docs justify the feature partly by quality degrading as conversations lengthen (compaction overview). See long-context degradation.
  • Cost and latency. Every turn re-sends the whole history, so cost grows with history size (Claude Code best practices, OpenAI conversation state).

Mechanics

A compaction step has four decisions: when to trigger, what to summarise (everything or all but the last k turns), how (prompt and model), and what to re-inject afterwards (instructions, files, skills, plan).

flowchart TD
  A["Agent turn: history grows"] --> B{"Input tokens ≥ trigger?"}
  B -- "no" --> A
  B -- "yes" --> C["Optional: warn the agent, flush notes to memory/files"]
  C --> D["Summarise older turns (keep last k verbatim)"]
  D --> E["New context = system + tools + summary block + recent turns"]
  E --> F["Re-inject durable state: CLAUDE.md, skills, todo/progress files"]
  F --> A

Synthesis: the diagram combines the trigger and block-replacement mechanics of the Anthropic and OpenAI APIs, the pre-clear memory warning from Anthropic’s context editing, and the re-injection behaviour of Claude Code; no single source shows all steps together.

Classic client-side compaction

The harness counts tokens, calls a model with a summarisation prompt, and rewrites its own message list. Examples:

  • MemGPT keeps a recursive summary at the head of a FIFO message queue. It warns the model at about 70% of the window so it can save facts. At the flush point it evicts about half the queue into the summary. These thresholds are example settings (Packer et al.). See Letta and agent memory.
  • LangChain SummarizationMiddleware takes a trigger and keeps the last 20 messages by default (LangChain middleware). See agent hooks and middleware.
  • Cognition recommends a separate compression model that distils history into key decisions and events for very long single-threaded runs. They fine-tuned smaller models for it and report it is hard to get right (Cognition).
  • Claude Code / Claude Agent SDK compact automatically near the limit. The SDK emits a compact_boundary system message (Agent SDK loop). A “Compact Instructions” section in CLAUDE.md steers what survives (How Claude Code works). PreCompact/PostCompact hooks exist in both Claude Code and Codex (Claude Code hooks, Codex hooks).

Anthropic’s SDK-level client compaction (compaction_control) is deprecated in the TypeScript and Ruby SDKs and was removed from the Python SDK in v1.0. Anthropic points tool-runner users to the server-side edit instead (context editing).

Server-side compaction: Anthropic

As of November 2026, the Messages API offers two beta modes, plus “write your own summariser” (overview):

On demandAt a token threshold
Beta headercompact-2026-09-04compact-2026-01-12
HowSeparate request with "compaction": {"type": "summarize"}; returns only a compaction block, stop_reason: "compaction"Edit compact_20260112 in context_management.edits; summary is written inside the request that crosses the trigger
Who decides whenYour codeThe API (trigger default 150,000 input tokens, minimum 50,000)
History afterwardsYou put the block first in messages and remove the summarised turnsAppend the response as usual; the API ignores everything before the last block
Keep recent turns verbatimYesVia pause_after_compaction and reinsertion
BackgroundYesNo

Sources: on-demand docs, threshold docs. Anthropic recommends on-demand wherever it is available (overview).

Details that matter in production:

  • The block is signed and readable. On-demand blocks carry a summary you can read plus a signature. Editing either returns a 400 error, and so does leaving summarised messages in front of the block (on-demand).
  • Custom prompts replace, not extend. instructions replaces the default summarisation prompt entirely; on demand it accepts up to 16,384 characters. The default asks for state, next steps and learnings (threshold, on-demand).
  • Billing hides in usage.iterations. Top-level token counts exclude the compaction pass, so cost tracking must sum the iterations. Carrying a block forward adds no compaction cost (threshold).
  • Caching. Put cache_control on the block and a breakpoint at the end of the system prompt, so the stable prefix stays cached across compactions (threshold). See prompt caching.
  • Thinking. Pre-compaction thinking blocks are not carried forward. Reinserting earlier assistant turns can fail the thinking prefix check, so strip their thinking or set prefix_mismatch_behavior: "drop_block" (threshold). See extended thinking.
  • Not combinable. compaction and context_management cannot be sent on the same request (on-demand).
  • Retention. Threshold compaction is ZDR-eligible except for Covered Models, because summaries round-trip statelessly (threshold). See data retention and residency.

Server-side compaction: OpenAI

The Responses API offers two modes (OpenAI compaction):

  • Automatic: context_management: [{type: "compaction", compact_threshold: N}]. When the rendered prompt crosses N tokens, the server compacts mid-stream, emits a compaction item and prunes. The guide’s examples use 200,000; no default is documented.
  • Explicit: POST /responses/compact takes a full window and returns a smaller one, statelessly. Pass the output on unmodified.

The compaction item is encrypted and opaque: unlike Anthropic’s block, you cannot read or audit what was kept. With manual input chaining you may drop items before the latest compaction item. With previous_response_id chaining you must not prune. Both modes work with ZDR when store=false (OpenAI compaction). In the OpenAI Agents SDK, OpenAIResponsesCompactionSession wraps this. Automatic compaction can hold a streamed run open for a few seconds, so it can be run manually instead (Agents SDK sessions).

TechniqueWhat is removedLossy?Cache impactSources
CompactionOlder turns, replaced by a summaryYes, judgement of the summariserInvalidates everything after the stable prefixAnthropic, OpenAI
Tool-result clearing / observation maskingOld tool outputs, replaced by a placeholder; calls and reasoning stayYes, but predictable and re-fetchableInvalidates from the cleared point; batch clearscontext editing, Lindenbauer et al.
Thinking clearingOld reasoning blocksYesInvalidates at the clearing pointcontext editing
Context resetEverything; a fresh agent starts from a structured handoff artifactOnly what the handoff omitsFresh contextharness design
External memory / notesNothing by itself; state is written outside and re-readDepends on what is writtenNeutralmemory tool, Manus
SubagentsExploration happens in a separate window; only a condensed result returnsYes, at the boundaryParent prefix untouchedAnthropic

Tool-result clearing

Anthropic’s clear_tool_uses_20250919 edit runs server-side before the prompt reaches Claude, while the client keeps the full history. Defaults: trigger at 100,000 input tokens, keep the last 3 tool uses, clear oldest first, leave tool inputs unless clear_tool_inputs is set, and exclude_tools for results that must survive. clear_at_least exists because each clear rewrites the cached prefix, so a clear should free enough tokens to justify the cache write. Before clearing, Claude gets a warning so it can save important results with the memory tool. Applied edits are reported in context_management.applied_edits (context editing, memory tool). LangChain’s ContextEditingMiddleware mirrors the defaults: 100k tokens, keep 3 (LangChain). SWE-agent collapses observations older than the last 5 to one line each (SWE-agent).

The strongest evidence on clearing versus summarising comes from Lindenbauer et al. On SWE-bench Verified with SWE-agent and Qwen3-Coder 480B, the raw agent solved 53.4% at $1.29 per instance. Masking old observations solved 54.8% at $0.61, and LLM summarisation solved 53.8% at $0.64. Summarisation also lengthened runs by about 15%. A hybrid (mask first, summarise only as a last resort) was cheapest (Lindenbauer et al.). Manus’s version of this idea is restorable compression: drop a page’s body but keep its URL, drop a file’s content but keep its path, so the agent can re-fetch (Manus).

Context resets vs compaction

Anthropic’s Labs team distinguishes the two. A reset clears the window and starts a fresh agent from a structured handoff. Compaction summarises in place, so the same agent continues: it keeps continuity but gives no clean slate. On Sonnet 4.5, “context anxiety” (wrapping up early as the model believed its limit was near) persisted under compaction, so their harness needed resets. Resets cost orchestration complexity, tokens and latency. With Opus 4.5 the behaviour largely disappeared, and the harness ran one continuous session on the Agent SDK’s automatic compaction (harness design). An earlier Anthropic post found compaction alone insufficient for multi-session app builds on Opus 4.5. Its fix externalised state into a JSON feature list, a progress log and git, read at the start of each session (long-running harnesses). See agent handoffs and durable agent execution.

Synthesis: the two posts are not in conflict. Compaction keeps a session coherent, while externalised artifacts make the work recoverable whether the next window comes from a compaction, a reset or a crash. Whether resets are needed is model-dependent and should be re-tested on each model upgrade.

Failure modes

FailureMechanismMitigation
Lost instructionsEarly detailed instructions get summarised away (Claude Code); on Anthropic’s API, mid-conversation system messages and per-message effort stop applying once summarised (on-demand)Keep durable rules in the system prompt / CLAUDE.md, which is re-injected every request (Agent SDK); see agent instruction files
Dropped detailThe default summary omits a fact a later turn needs (overview)Custom instructions naming what to retain; keep recent turns verbatim; flush to memory before compaction (memory tool)
Truncated skills/attachmentsClaude Code re-attaches only the first 5,000 tokens per skill (25,000 total) after compaction (skills); images, documents and fetched URLs inside the summarised range are gone on the API (on-demand)Put critical rules at the top of skills; restate or re-upload what is still needed
ThrashingOne huge output refills the window right after compacting; Claude Code stops with an error (Claude Code)Cap tool output size; page or store large results in files
Trajectory elongationSummaries make agents run about 15% longer (Lindenbauer et al.)Prefer masking first; set turn and budget caps (agent loop)
Premature victory / context anxietyA fresh or compacted context sees progress and declares done, or wraps up early (long-running harnesses, harness design)External feature list with pass/fail status; evaluator agent; resets on affected models
Summariser misbehavesNo block returned: truncated (max_tokens), no room (model_context_window_exceeded), tool call instead of summary, or refusal (on-demand)Branch on stop_reason; tell the summariser not to call tools; leave headroom below the window
Hidden costThe compaction pass is billed but excluded from top-level usage; each compaction busts the cache after the prefix (threshold)Sum usage.iterations; cache the system prompt and the block; monitor in LLM observability
Unauditable stateOpenAI compaction items are encrypted (OpenAI)Log your own transcript pre-compaction; keep critical state in files you control
Broken serialisationRe-sending a block with extra null fields or edited content returns a 400 (threshold, on-demand)Round-trip the block exactly as returned

Practical guidance

Synthesis, drawing on the sources above:

  1. Clear before you summarise. For tool-heavy agents, first mask or clear old tool results (keep 3 to 10 recent observations). Reach for an LLM summary only when that is not enough. The evidence shows masking matches summarisation at lower complexity (Lindenbauer et al., context editing).
  2. Trigger well below the window. Leave room for the summarisation pass and the next turns. Anthropic’s threshold default is 150k, and its minimum is 50k. Choose a trigger by quality, not only capacity, since degradation starts long before overflow (Chroma).
  3. Make durable state external. Keep instructions in the system prompt or instruction files, the plan in a todo file, progress in a log, and work in git. The summary then only has to carry the narrative (long-running harnesses, Manus).
  4. Write a domain-specific summary prompt. Spell out what must survive: decisions and their reasons, open questions, file paths and IDs, failed approaches, user constraints. Forbid tool calls (on-demand).
  5. Keep recent turns verbatim. Use on-demand compaction with kept turns, or the threshold mode with pause_after_compaction (overview).
  6. Use hooks. A PreCompact hook can persist state or a transcript before the window shrinks (Claude Code hooks).
  7. Evaluate it. Replay long traces with compaction forced early, check that specific facts survive and the task still completes, and track turns and cost per task. See agent evaluation.
  8. Prefer /clear or a new session over compaction between unrelated tasks (Claude Code best practices). Use subagents for exploration so the parent window never fills (Anthropic).

Sources

Suggest changes →