The idea

When you ask an AI coding tool to review a change, its request may include your instructions, source files, test output, project rules, and earlier chat. That material is represented as tokens: model-specific units used for input and output text. A model can consider only a finite amount of material at once. That limit is its context window.

Token optimization is the practice of fitting the most useful information into a request while avoiding unnecessary context. It is not simply making every prompt shorter. The real question is: what does this task need now to produce a useful answer?

Plain English: Treat the AI request like a focused work packet. Keep the requirements and evidence needed for this job. Leave out material that will not affect the decision.

Start by measuring rather than guessing. Token counting estimates the number of input tokens before you send a request. Anthropic documents this for managing prompt length, cost, rate limits, and model routing. Its documentation also says counts are estimates and can differ across model generations, so count against the model you intend to use. Source

Then divide material into two groups. Put short, stable rules that apply to nearly every task in the always-available context. Put a long API guide, release checklist, or migration notes into on-demand context: reference material loaded only for a relevant task. For example, Claude Code documents CLAUDE.md as automatically loaded, while skills can expose a description first and load their full content when used. It also warns that extra material can fill the context window or add noise. Those are Claude Code-specific mechanisms, not universal behavior of every AI tool. Source

If many requests begin with the same large, stable material, prompt caching may help. This provider feature reuses a stable prompt prefix on later requests. Anthropic documents it for repeated prompts, large context, examples, and long conversations. It does not make an oversized request fit into the context window. Source

After a request, inspect usage metadata when your provider exposes it. Google documents token-counting and token-usage metadata after generation; this lets you compare an estimate with observed usage. Source

A small example

Example — hypothetical code review: You ask an AI tool to review a change to an authentication endpoint. Your usual request includes repository conventions, the changed endpoint and tests, a 40-page API style guide, deployment runbooks, and a historical incident log.

The endpoint, tests, and a few repository conventions affect this review. Keep them in the request. The API style guide may matter if it defines this endpoint's contract, so load it only when that is true. The deployment runbook and incident log do not help the code-review question, so do not attach them.

This is not a rule to remove everything. If the incident log defines a security constraint that the change must preserve, it belongs in the work packet. Token optimization is selecting relevant evidence, not blindly minimizing text.

Try it

Spend five minutes on one recurring AI request.

  1. Copy its sections into three buckets: always needed, needed for this task, and optional reference.
  2. Count the input with the target model's tool or inspect the tool's reported usage, if available.
  3. Keep only core project rules in the always-needed bucket.
  4. Run one task with the optional material omitted or loaded only when relevant.
  5. Compare the answer. Restore any constraint, definition, or example that the tool needed to do the job correctly.

Write down what changed. You are building a small context budget for a real workflow, not chasing the lowest possible number.

Watch out

A smaller request can produce a worse answer. Never remove requirements just because they are long. Token counts are not universal: the same text can count differently for another model or tokenizer generation. Source

Also separate two goals. On-demand material reduces what is loaded for a particular task. Prompt caching can reuse repeated stable input where a provider supports it. Neither replaces judgment about relevance, and caching does not increase the model's context window. Finally, tool features vary. A workflow based on CLAUDE.md, skills, or isolated subagents should be treated as a tool-specific design, then adapted to the AI product you actually use. Source