Context Engineering
Why it matters
Deep Dive
Context engineering treats the context window as a managed resource with a budget, a lifecycle, and known failure modes. A typical LLM call assembles its context from many sources: a system prompt that defines behavior, few-shot examples, documents pulled in by retrieval, tool definitions and their results, long-term memory, and the running conversation history. The discipline is deciding what earns a place in that window, in what format and order, when to refresh or compress it, and how to keep irrelevant or wrong information out. Where prompt engineering asks how to phrase the instruction, context engineering asks what the model should know by the time the instruction arrives.
The Context Budget
Context windows have grown from a few thousand tokens to hundreds of thousands and beyond, but the space is never free. Every extra token adds latency and cost to every call, and attention does not scale gracefully: research on long-context models shows that information buried in the middle of a long prompt is used less reliably than information near the beginning or end, a pattern known as "lost in the middle." Practical context engineering therefore means budgeting. Teams compress old conversation turns into running summaries, trim verbose tool outputs down to the fields that matter, deduplicate repeated boilerplate, and move reference material out of the prompt and into retrieval so it is fetched only when relevant. The goal is the smallest context that still contains everything the model needs for the current step.
Assembling Context at Runtime
Modern applications build context dynamically rather than writing it by hand. A RAG pipeline retrieves documents relevant to the user's question and injects them at call time; a memory system carries durable facts about the user across sessions; tool use and protocols like MCP let the model pull in live data mid-conversation, with each result appended to the context for the next step. Agent loops take this furthest: every observation, action, and intermediate result lands in the context, so the context effectively becomes the agent's working state, its short-term memory and plan rolled into one. This is why context engineering shows up so strongly in agent design — deciding what to persist, what to summarize, and what to drop across a fifty-step run is often the difference between an agent that stays on task and one that drifts.
A Bigger Window Is Not the Fix
A common misconception is that million-token context windows make context engineering unnecessary — just dump everything in and let the model sort it out. In practice, long context introduces its own failure modes. Context rot describes the gradual degradation of performance as the window fills: accuracy on retrieval-style questions drops as input length grows, even on models advertised for very long context. Context poisoning is nastier: once a hallucinated claim, a stale fact, or an injected instruction enters the context, the model tends to treat it as ground truth and build on it, and agents that write their own outputs back into the context can amplify one early mistake across every later step. Related failure modes include context distraction, where irrelevant material pulls the model off task, and context clash, where contradictory information in the window produces hedging or arbitrary choices. Curating the inputs, validating what gets written back, defending against prompt injection, and periodically resetting the context are the standard countermeasures.
From Prompt Engineering to Context Engineering
Prompt engineering, the older discipline, focused on phrasing: instruction wording, few-shot example selection, role prompts. It assumed a human writing a single prompt against a mostly blank window. As applications grew into pipelines and agents, the bottleneck moved — the hard part became managing the flow of information into a window shared by instructions, data, tools, and history, most of it assembled by software rather than typed by a user. The term context engineering caught on in 2025 among practitioners to name exactly this shift, and it stuck because it described what teams were already doing. Prompt craft still matters, and in-context learning remains the mechanism everything rides on, but phrasing is now one component inside a broader engineering surface that also covers retrieval quality, memory design, tool output formatting, and context lifecycle management.