The idea

A long AI-assisted coding task creates a growing history. The history may include your requests, model replies, file changes, test results, and tool output. All of this competes for space in the model’s context window: the bounded amount of information it can consider while producing a response. In long-running work, the available context can eventually fill. Anthropic describes server-side compaction for conversations and agentic workflows approaching a context limit.

Context compaction is a way to continue without carrying every old event forward. It replaces a long conversation state with a smaller representation intended to preserve the state needed for the next turn. OpenAI describes compaction as reducing context size in long-running interactions while preserving state needed later. Source: OpenAI.

Plain English: Compaction is a handoff note for the next stage of one task. It is not a perfect transcript, and it is not permanent memory.

Think of it as reducing a verbose debugging log to the current bug, evidence, decisions, and next check. The analogy has limits: a compacted state is machine-oriented data for continuing model work, not necessarily a human-readable project note. OpenAI says compacted output should be treated as opaque state and passed on unchanged rather than manually edited. Source: OpenAI.

Compaction and long-term memory solve different problems. Compaction continues a current run. Memory can retain reusable workflow lessons across runs. OpenAI makes this distinction in its agent guidance. Source: OpenAI.

A small example

Example: You ask an agent to diagnose a failing integration test. Over many turns, it reads files, changes a configuration value, runs tests, and collects repeated logs. At a useful milestone, the active history could be compacted into a state such as:

  • Goal: make checkout integration pass without changing production behavior.
  • Evidence: the test fails after the mock payment response is parsed.
  • Changes tried: configuration change did not fix the failure.
  • Next step: inspect the parser’s handling of the mock response.

The next request continues from this smaller state instead of requiring every log line and old reply. Some API systems can perform this automatically near a threshold; OpenAI documents both server-side compaction through context_management and compact_threshold, and a separate compaction endpoint. Source: OpenAI. Anthropic also describes server-side compaction for conversations and agentic workflows approaching a context limit. Source: Anthropic.

The list is an explanatory example, not a prescribed API format. Provider names, triggers, model support, and compacted-data formats differ.

Try it

Take about five minutes after a debugging milestone. This is a hypothetical exercise; you do not need a compaction API.

  1. Choose a task with several agent turns.
  2. Write four bullets: the goal, confirmed evidence, changes already tried, and the next action.
  3. Put crucial details in a durable, checkable artifact, such as a failing test, issue note, source file, or command output saved with the work.
  4. Start a fresh prompt using those bullets. Ask the agent what it would check next.

Then compare its proposal with the original task. If it misses a constraint, add that constraint to the handoff note or point it to the artifact. This is context selection: deciding what the next turn actually needs.

Watch out

Compaction can omit or distort details. Do not treat it as the sole record of a decision, source, or test result. Keep important facts where you can inspect and verify them: in code, tests, documents, or other artifacts.

A bigger context window does not automatically make every answer more accurate. Anthropic notes that larger token counts can bring accuracy and retrieval challenges. Source: Anthropic. Compaction also does not excuse oversized inputs. Remove stale requirements, avoid repeatedly sending huge outputs, and keep the next action explicit.

Finally, do not use compaction when a task is a small one-off question. It is most useful when one task must survive many steps. Treat it as a controlled continuation mechanism, then verify the work with the same engineering evidence you would use without AI.