Context engineering is the practice of deciding exactly what goes into a model's context window at each step: instructions, tools, retrieved documents, memory, and history. Karpathy popularised the term in June 2025. It matters because more tokens don't mean better answers, models lose recall as input grows, a problem called context rot.
In 2026, million-token context windows are ordinary. Claude Opus 5 ships with 1M tokens by default, GPT-5.6 Luna offers about 1.05M, and Gemini 3.7 Flash sits at roughly the same. And yet agents still lose the thread halfway through a long task, contradict themselves, and forget instructions they were given an hour ago.
The reason isn't window size, it's what's inside the window. That's the idea behind context engineering, the discipline of curating everything a model sees at the moment it makes a decision. This guide explains what it is, where it came from, and the techniques that actually work, without pretending prompt engineering has vanished.
What Is Context Engineering, in Plain English?
Andrej Karpathy gave the term its push in June 2025, framing it as the craft of filling the context window with the right information for the next step, no more and no less. Anthropic's engineering team describes it as the natural progression of prompt engineering.
Anthropic's shorthand for the goal is finding "the smallest possible set of high-signal tokens" that makes your desired outcome most likely. The word that matters is smallest, not largest.
Think of it as packing a bag for a single trip. You don't bring the whole house, you bring what this specific journey needs, and you repack at every stop.
How Is It Different From Prompt Engineering?
Prompt engineering is about wording the instruction well. Context engineering is about everything the model reads when it acts: the system prompt, the tool definitions, the retrieved documents, the memory, the conversation history, and the output of every tool it has already called.
For a one-shot question, the two are nearly the same. For an agent that runs for dozens of turns, they diverge sharply. A perfectly worded prompt is still doing its job inside a window that's now stuffed with stale tool output and half-relevant documents.
Prompt engineering asks how to say it. Context engineering asks what should be in the window right now.
Why Doesn't a Bigger Context Window Solve This?
Because capacity and attention are different things. Anthropic and Chroma both describe context rot: as the number of tokens grows, a model's ability to accurately recall what's in them drops. Chroma's researchcontext-rot tested 18 models and found degradation that was uneven, showed up even on simple tasks, and got worse when distractor content was present.
That's why a million-token window isn't a license to paste everything. It raises the ceiling on what can fit, not the quality of what the model does with it.
The practical consequence is counterintuitive: a shorter, cleaner context frequently beats a longer, noisier one, even when the longer one technically contains more of the answer.
What Actually Fills Up a Context Window?
More than the visible chat. The system prompt, every tool definition, the full message history, and every tool result all count against the same budget.
In agent work, tool results are usually the biggest culprit. A single web search or file read can dump thousands of tokens that the agent needs once and then carries around for the rest of the run.
Tool definitions are the sneaky second. Lots of overlapping, verbosely described tools quietly eat budget on every single turn, and confuse the model about which one to use.
What Are the Core Techniques?
LangChain's framingcontext-engineering-for-agents/ groups the work into four operations. Write means saving information outside the window, in a scratchpad or notes file, so it isn't lost. Select means pulling in only what's relevant, through retrieval or choosing which tools to expose. Compress means summarizing or trimming what's already there. Isolate means splitting work across separate contexts, such as sub-agents, so no single window accumulates everything.
Anthropic's engineering guideeffective-context-engineering-for-ai-agents names three techniques for long-horizon tasks: compaction, structured note-taking, and multi-agent architectures. They map neatly onto compress, write, and isolate.
You don't need all of them. Most teams get the biggest early win from just two: cleaning up tool results and compacting long sessions.
What Does Compaction Do, and What Does It Cost You?
Compaction takes a conversation nearing its limit, summarizes it, and starts a fresh window seeded with that summary. Anthropic calls it typically the first lever to pull for long-term coherence.
The cost is that summaries are lossy. An over-aggressive one can drop a subtle detail that mattered three steps later. The advice from Anthropic's guidance is to tune the compaction prompt deliberately: start by maximizing recall so nothing important is lost, then iterate toward precision by trimming what's redundant.
Reported results are encouraging but come with caveats. In an internal evaluation, Anthropic reported that automatic clearing of stale tool results cut token consumption by 84% across a 100-turn web search task, while letting workflows finish that would otherwise have failed from context exhaustion. A separate academic experiment shrank a five-ticket support pipeline from about 208K to 86K tokens, roughly a 59% reduction. Treat both as demonstrations of the technique, not guarantees for your workload.
How Does Memory Fit In?
Structured note-taking and memory tools let an agent write important facts to files outside the window and read them back later, so a long task survives resets and compaction. Anthropic introduced context editing and a memory tool alongside Claude Sonnet 4.5 on September 29, 2025.
Memory brings its own risk. Whatever gets saved can steer future runs, which is exactly why a poisoned note can be dangerous. Validate what gets written to long-term memory the same way you'd validate user input.
How Do You Practice Context Engineering Today, Even Without Building Agents?
Start a new chat for a new task instead of dragging an old one along. Put the key facts and the goal at the start, and restate them if a conversation runs long.
Paste only the relevant excerpts of a document, not the whole file. Remove contradictory or duplicated information before you ask. When a long session gets sluggish, ask for a summary of decisions so far and carry only that into a fresh chat.
These are practical heuristics drawn from how the research describes failure, not rules validated by a single benchmark. Test them on your own tasks.
What Should Developers Audit First in an Agent's Context?
Measure before you optimize. Count tokens by component: system prompt, tool definitions, history, and tool results. Most teams discover one component is doing most of the damage.
Then prune stale tool results, trim and de-duplicate tool descriptions, load documents just in time rather than up front, and set a compaction threshold. Finally, test with long sessions, since problems that never appear in a five-turn demo show up reliably at turn fifty.
Is Prompt Engineering Actually Dead?
No, and the framing of one skill "replacing" another oversells it. Instructions still matter, they're simply one slice of a bigger picture. Context engineering is a superset, not a successor that makes the old skill worthless.
It's also fair to note the vocabulary is still settling. Researchers use terms like context rot, context degradation, and attention dilution for overlapping ideas, and parts of the trend are rebranding. What's genuinely new is the shift from writing one good prompt to managing a window across many steps, which is where agents live.
Quick Verdict
Bigger windows didn't fix long-running AI, because the limit was never space, it was attention. Context engineering is the practical response: keep the window small, relevant, and fresh, write important things down outside it, and isolate work so nothing carries everything. Learn it if you build agents, and borrow its habits if you simply use AI heavily.
Frequently Asked Questions
What is context engineering?
The practice of deciding exactly what information goes into a model's context window at each step, including instructions, tools, retrieved documents, memory, and history, so the model has the right information and nothing that distracts it.
How is context engineering different from prompt engineering?
Prompt engineering is about wording the instruction. Context engineering covers everything the model reads when it acts, which matters most for agents running many steps.
What is context rot?
The drop in a model's ability to recall information as its input grows. Chroma's research measured it across 18 models and found it gets worse when distractor content is present.
Does a bigger context window fix the problem?
No. It raises how much fits, but not how well the model uses it. A shorter, cleaner context often performs better than a longer, noisier one.
What are the main context engineering techniques?
Write information outside the window, select only what's relevant, compress what's there through compaction, and isolate work across sub-agents. Anthropic highlights compaction, structured note-taking, and multi-agent architectures.
