What is context engineering?
Context engineering is selecting and maintaining all information a model sees when reasoning: instructions, tools, external data and conversation history, not just writing a command. According to a post by Anthropic's engineering team (posted 29/9/2025, read 25/10/2026), this is iterative because each time you feed data into the model, you must choose. This post explains what context engineering is, why context is a finite resource, four main techniques, when to use them and OPCAIOS's experience.
Context is a limited attention budget: choose little but right. Diagram: OPCAIOS.
What is context engineering and how does it differ from prompt engineering?
Prompt engineering focuses on writing and arranging instructions, especially the system prompt. Context engineering is broader: managing everything in the context window at each step, including tools, retrieved data and history. Both complement each other; prompt is one part of context.
With a chatbot answering in one turn, writing a good prompt is usually enough. With AI agent running many steps, context grows over tool results and history, so the question shifts from "what to write" to "what to keep, what to drop at this step".
Why is context a finite resource?
Models lose accuracy as the context window fills; the Anthropic paper calls this effect "context rot". Each token added draws from a limited attention budget, and according to the paper the decline happens gradually rather than as a cliff.
Practical consequence: a large window does not mean you should cram everything in. Anthropic's documentation on context window also says more context is not automatically better, and choosing important content matters as much as remaining capacity. See also token and context window.
Four main techniques of context engineering
The Anthropic paper lists four techniques: compaction, structured notes outside the window, sub-agents with clean context and just-in-time data loading. Each technique solves one kind of problem in long tasks.
| Technique | What it does | When to use |
|---|---|---|
| Compaction | Summarize conversation about to fill, then restart from the summary; delete old tool results | Very long sessions |
| Structured notes | Agent saves notes outside the window, reloads when needed | Tasks many hours, many sessions |
| Sub-agent | Hand off narrow work to an agent with clean context, receive a summary back | Separate search, research work |
| Just-in-time loading | Keep light references (paths, queries), load data only when needed | Large data stores |
The paper also says practical ways often blend: preload some and retrieve more as needed. Looking up separate documents often uses RAG and vector database.
When to pay attention to context engineering?
Pay attention when the task takes many steps, uses many tools or many documents, and when you see the agent "forget", repeat or wander. For short one-turn work, prioritize writing a clear prompt first.
| Symptom | Way to handle |
|---|---|
| Agent forgets a decision made | Structured notes, reload at session start |
| Old tool results cause confusion | Delete or summarize used results |
| Cost and latency grow over time | Compress context; see prompt caching |
| Search in large document store | Load just-in-time instead of cramming all |
OPCAIOS's experience
How we work with agents is a form of manual context engineering: a guide file read at session start, a system state file as source of truth, and a rule "only read files matching the work at hand, don't read everything at once" (just-in-time loading). Important results from each session are logged to a journal to be read back in the next session (notes outside the window).
Limits: this is a work practice, we have not measured against control whether it cuts errors by how much. And the state file itself can bloat or go stale if not cleaned, so we need to rotate and check directly before trusting. See the AI system I use to run work alone.
Frequently Asked Questions
Does context engineering replace prompt engineering?
No. Prompt engineering is a part of context engineering. You still need clear instructions; context engineering additionally handles the data and accompanying tools.
Does a 1 million token context window make context engineering unnecessary?
No. Anthropic's article doesn't expect a large context window to self-solve accuracy degradation; content quality still matters more than volume.
What is context rot?
It's the phenomenon where a model's accuracy and recall ability gradually decrease as the number of tokens in the context increases. It's the term used in Anthropic's article and their documentation.
How do I start without needing code?
Start with a habit: summarize your decisions at the end of a session into a file, paste it back at the start of the next one. That's already external context window notes.
Are there practical training materials?
You can start with our AI application training service to design workflows with agents.
Source: Anthropic Engineering, "Effective context engineering for AI agents" (29/9/2025) and Anthropic's "Context windows" documentation, read on 25/10/2026. Technical table summarized from the original article. Experience is from OPCAIOS, not yet validated.
Translated from Vietnamese with AI. Read the original.
Want to apply this in your business?
Book a free 30-minute call (Vietnam time, in English or Vietnamese).