AI coding tools handle context window limits by deciding what the model never sees. They index the repository and retrieve a few relevant slices rather than loading everything, read files on demand instead of upfront, summarise or compact the conversation when it approaches the ceiling, and keep standing project rules in a file so the important instructions survive a reset. The window itself has grown enormously, but the engineering effort has not gone away, because accuracy falls as the window fills and every extra token costs money and time. Context management is the product; the model is the part that does the typing.
What is actually in the window
A context window is the fixed token budget a model can attend to at once. IBM describes it as the model’s working memory: it sets how long a conversation can run before earlier details fall out of reach, and how large a document or code sample can be processed in one go. That framing is fine for a chatbot. For a coding agent it undersells the problem, because the budget is shared by far more than your messages.
In an agentic session the window holds the system prompt, your instructions, the project rules file, the contents of every file the agent opened, the output of every command it ran, the stack traces from failing tests, and the full back and forth so far. Pickuma’s write-up on context windows in coding agents puts it plainly: once that budget fills, the model starts forgetting earlier instructions, inventing APIs that do not exist, and losing track of the file structure it was working in. If you have watched an agent confidently call a method you have never had, the cause is often not the model being weak but the context being wrong, a pattern we looked at in why AI coding tools invent APIs that do not exist.
A single run of a test suite can produce thousands of tokens of output that matter for ten seconds and then sit in the window for the rest of the session. That is the real pressure. Code is not usually what fills a context window. Noise is.
Bigger windows did not solve it
The headline numbers have moved fast.
Two things stop that being the end of the story.
The first is that filling the window degrades the answer. Anthropic’s own documentation acknowledges context rot: as token count grows, accuracy and recall fall away. The practical version, as the guide to context windows for AI coding puts it, is that a model holding a million tokens of mediocre context performs worse than one holding fifty thousand tokens of carefully chosen code. More room is not more understanding.
The second is arithmetic. Attention compares every token with every other token, so a sequence of a thousand tokens means a million comparisons and ten thousand tokens means a hundred million, as MindStudio’s explanation of the model context window sets out. A tool that solves a task by loading everything is a tool with an expensive habit, which is one reason the bill for these tools lands well above the seat price, covered in what AI coding tools really cost beyond the seat.
Prompt caching takes some of the sting out. Anthropic’s approach lets developers place explicit cache breakpoints, and cached tokens are then read at a discount, according to Finout’s guide to Anthropic pricing. That makes a large, stable project preamble affordable to resend. It does not make an undisciplined session cheap, because the part that changes every turn is the part that is never cached.
The four moves every tool makes
Strip away the branding and the techniques converge.
Selection. The agent reads the files that look relevant to the task rather than ingesting the repository. Everything else in context management is downstream of getting this first decision roughly right.
On-demand reading. Rather than assembling a complete picture before starting, the agent reads one file, decides what to do, then reads another. It is slower in wall-clock terms and much cheaper in tokens, and it keeps irrelevant code out of the window entirely.
Compaction and summarisation. When the session approaches the ceiling, the tool condenses earlier turns into a shorter record: decisions and current state kept, verbatim command output discarded. Claude Code documents this explicitly in its context window guide, and the behaviour is available both automatically and on demand.
Durable notes. A project conventions file loaded at the start of every session carries the standing rules, the architecture in outline and the things the agent must not do, without the agent having to rediscover them by reading forty files. This is the layer most teams under-invest in, and it is the cheapest one to fix.
Underneath those four, agents working on bulk input also use chunking with overlapping windows, processing each chunk independently and merging the results. That suits log analysis, document extraction and scanning a codebase for a pattern. It suits refactoring across coupled modules much less well, because the dependency the change hinges on is often in the chunk the model is not currently looking at.
Cursor: index first, then discover
Cursor is worth describing in detail because it shows both halves of the problem being handled separately.
The first half is retrieval. On opening a project, Cursor embeds the repository’s files with its own model and stores the vectors in a per-codebase namespace.
The second half is what happens to retrieved material. The naive approach is to paste the top matching chunks into the prompt, which fills the window with near-misses. Cursor has moved towards dynamic context discovery, treating the file as the unit of context management rather than the chunk. Tool responses too large to inject are written to temporary files, and the agent is handed the path. It can then read the file, tail it, or grep it, depending on what it is trying to establish. Nothing is lost to truncation, and nothing enters the window unless the agent decides it needs it.
That design is a direct answer to the cost curve above. The agent pays tokens for the decision to look, not for the possibility of needing to.
Repository maps, and giving the model shape instead of substance
Aider takes a different route to the same end with its repository map: rather than file contents, the model receives an outline of the codebase, the names and signatures of the things that exist and how they relate. It is a map, not the territory. For a question like “where should this new handler go and what does it need to call”, a map is often enough, and it costs a fraction of the tokens that reading the relevant files would.
The general lesson holds across tools. Much of what a model needs in order to make a good decision about code is structural, and structure compresses well. Implementation detail does not compress and should be fetched only when the work has narrowed to the file that contains it.
The numbers can move without telling you
Treat vendor context limits as a current reading rather than a guarantee.
Published figures also disagree with each other. Aggregator sites list the same Gemini Pro model variously at 1,048,576 tokens and at ten million, which tells you something about how much weight to put on any single table. If a limit matters to your workflow, measure it against the behaviour you get rather than against a number on a comparison page, and build the workflow so that an unannounced cut degrades output quality rather than breaking a pipeline at two in the morning.
Budgeting the window yourself
The tools manage context for you. You can still manage it better, and the gains are not marginal.
Start by reserving room. One sensible rule of thumb is to leave thirty to forty per cent of the window for the agent’s accumulated work: on a 200,000 token model, roughly 80,000 for the system prompt and project context, 80,000 for tool results as they pile up, and 40,000 for reasoning and output. If your project preamble alone consumes half the budget, the agent has nowhere to think.
Then keep tasks narrow. A session scoped to one change stays inside the clean part of the window, where recall is still good. This is the same argument as small iterations beat big rewrites in vibe coding, arriving from the token side rather than the review side. Starting a fresh session for an unrelated task costs you the warm-up and saves you the rot.
Put the standing rules in a file rather than in chat. Instructions typed into a conversation are the first thing compaction flattens. Instructions in a project context file are reloaded every session, and they are also reviewable by the rest of your team, which conversation history is not.
Finally, write prompts that carry the context the tool cannot infer, which is a skill in its own right and the subject of how to write a clear prompt for an AI coding tool. A prompt that names the files, the constraint and the expected shape of the result saves the agent an exploratory read that would otherwise have cost you ten thousand tokens and left half of them in the window for the rest of the session. The mechanics behind all of this, from prompting through to the review loop, are set out in how vibe coding works.
Written with AI assistance and reviewed under our editorial policy. Vibe coding changes quickly; details reflect sources current on 2026-10-12. Some passages are the author's opinion. Spotted an error? tell us.
Frequently asked questions
- Does a bigger context window mean I can skip context management?
- No. Anthropic's documentation acknowledges context rot, where accuracy and recall degrade as the token count grows, and a model holding a million tokens of loosely related material performs worse than one holding a well-chosen fifty thousand. Larger windows raise the ceiling on what is possible; they do not improve what the model does with a window full of noise.
- What is compaction, and when should I trigger it myself?
- Compaction replaces earlier conversation turns with a summary that keeps decisions and current state while discarding verbatim command output, and Claude Code documents it as both an automatic and a manual operation. Triggering it yourself is worth doing when you have just finished a noisy step, such as a long test run, and are about to start a distinct piece of work.
- Why does context usage cost more than the token count suggests?
- Attention compares every token with every other token, so cost and latency grow quadratically with input length rather than linearly. Prompt caching offsets some of this, with Anthropic's cached tokens read at around a ninety per cent discount, but caching only helps with the stable part of the prompt, not the part that changes every turn.
- Can I rely on a tool's advertised context limit staying the same?
- Not safely. OpenAI reduced the Codex CLI context window to a 272,000 token cap in mid-2026, a cut of roughly twenty-seven per cent delivered server-side with no version bump or announcement, and it surfaced only when long-context operations started failing. Build workflows that degrade gracefully if the ceiling drops rather than ones that depend on a specific number.