Context engineering

Why it matters

A model only knows what is in front of it. Everything it can use to answer has to fit inside one working space called the context window: your instruction, the earlier turns of the conversation, any documents pulled in, the descriptions of tools it can call. Context engineering is the practice of deciding what goes into that space each time the model runs, and what is left out. As soon as you move past a one-off chat and start building something that has to work reliably, this becomes the job that decides whether the model has what it needs, or is guessing.

The term is new but the pressure behind it is not. Simon Willison dates its rise to mid-2025, when Shopify’s Tobi Lütke used it and Andrej Karpathy amplified it. Karpathy’s line has stuck: context engineering is “the delicate art and science of filling the context window with just the right information.” The word caught on because “prompt engineering” had come to mean, to most people, just typing into a chatbot, and a bigger name was needed for the real work of assembling everything the model sees.

What is context engineering?

Context engineering is the practice of designing everything a model sees at the moment it runs: instructions, retrieved documents, tool definitions and conversation history. Anthropic describes it as “the set of strategies for curating and maintaining the optimal set of tokens (information) during LLM inference,” and draws the line with prompt engineering cleanly: prompt engineering is about “writing and organizing LLM instructions,” while context engineering is about managing the whole set of information the model has, across turns. LangChain’s Harrison Chase puts the practical version as “building dynamic systems to provide the right information and tools in the right format such that the LLM can plausibly accomplish the task.”

The key word is dynamic. A prompt is something you write once. Context is assembled fresh on every turn, often by code: pulling the right documents, trimming old history, deciding which tool descriptions to include. Getting that assembly right is the discipline.

What is in the context window?

The context window is the model’s whole field of view for one run, and context engineering decides what fills it. The usual ingredients are:

• The system prompt and instructions: the standing rules and the task at hand.
• Conversation history: the earlier turns, in full or summarised.
• Retrieved documents: facts pulled in for this specific question, typically through retrieval-augmented generation (RAG).
• Tool definitions: the descriptions of the tools the model may call, so it knows what is available and how to use it.
• Memory: notes carried over from earlier sessions or runs.

Every one of these competes for the same limited space, which is why context engineering is as much about what to leave out as what to put in.

Why context is a finite resource

The reason context engineering is hard is that the window is not free space. Anthropic argues that “context must be treated as a finite resource with diminishing marginal returns.” More is not better past a point. The company points to a pattern researchers call context rot: “as the number of tokens in the context window increases, the model’s ability to accurately recall information from that context decreases.” Stuff the window with everything that might be relevant and the model gets worse at finding the part that matters, not better. So the goal is not the most context but the smallest set of tokens that still does the job.

How do you engineer context?

The techniques are mostly ways to keep the window small and relevant while the work grows. Anthropic groups the main ones. Compaction summarises a long conversation into a shorter form so the thread can continue without carrying every earlier token. Structured note-taking, or memory, lets an agent write durable notes to a file and pull them back later, keeping information without keeping it in the window. Just-in-time retrieval loads data at the moment it is needed rather than pre-loading everything up front. And sub-agent architectures hand a focused task to a separate agent that does the work in its own context and returns only a condensed result. All four share one aim: give the model exactly what this step needs, and nothing that will crowd it out.

How is it different from prompt, harness and loop engineering?

Context engineering is one of four related layers, and they are cleanest kept apart by what each designs. Prompt engineering designs the wording of a single instruction. Context engineering designs what information enters the window on a given turn, the whole set the model sees, of which your instruction is one part. Harness engineering designs the system an agent operates within: its tools, sandbox, guardrails and success criteria. Loop engineering designs the autonomous repetition, how an agent iterates without a human prompting each turn.

The pair people most often confuse is context and harness, and Hugging Face gives the sharpest test: “Harness controls the loop; context controls the input to each loop iteration.” A tool’s description, the text that tells the model the tool exists, is context, because it enters the window. The tool itself, its execution and the sandbox it runs in, is harness. Some writers nest the layers rather than treat them as peers: LangChain calls prompt engineering “a subset of context engineering.” Either way the direction is the same, from wording one line, to assembling everything the model sees, to building the system around it.

Context engineering in practice

A support assistant makes the practice concrete. A customer asks a question; before the model answers, code assembles the context: the system prompt with the company’s rules, the last few turns of the chat, the two or three help-centre articles retrieved for this exact question, and the definitions of the tools the assistant can use, such as looking up an order. What is deliberately left out matters as much: the other nine hundred help articles, the full chat history, tools irrelevant to this request. Do that assembly well and a capable model answers accurately. Do it badly, flooding the window or omitting the one needed article, and the same model guesses or hallucinates. The model barely changes; the context does all the work.

Frequently asked questions

What is the difference between prompt engineering and context engineering?

Prompt engineering is about the wording of your instruction. Context engineering is about everything the model sees on a given turn: the system prompt, the conversation history, retrieved documents and tool descriptions, and how that window is assembled each time. Your prompt is one ingredient in the context. LangChain goes as far as calling prompt engineering a subset of context engineering, because as systems grow, getting the whole context right matters more than the phrasing of one line.

Isn’t more context always better?

No. Anthropic treats context as a finite resource with diminishing returns, and points to context rot: past a certain size, adding tokens makes the model recall the relevant ones less reliably, not more. The aim is the smallest set of information that still lets the model do the task, not the largest. Padding the window usually hurts.

How does context engineering relate to RAG?

Retrieval-augmented generation is one of the main tools of context engineering. RAG fetches documents relevant to the current question and places them in the window so the model can use facts it was not trained on. It answers the “what goes in” question for external knowledge. Context engineering is the broader discipline that also decides how much history to keep, which tools to describe and when to summarise.

Is context engineering the same as harness engineering?

No, though they sit next to each other. Context engineering governs what enters the model’s window each turn. Harness engineering governs the system the agent runs within: the tools themselves, the sandbox, the guardrails and the stopping criteria. A tool’s description is context, because it enters the window; the tool and its execution are harness.

Where should I start with context engineering?

Start by listing what the model actually needs to answer well, then make sure exactly that reaches the window and little else. Add retrieval for the facts it lacks, keep only the history that matters, and describe just the tools this step uses. When answers degrade, check whether the window is missing something needed or drowning in something irrelevant, both are common, and both are context problems.

Sources

• Anthropic: Effective context engineering for AI agents (Sept 2025) – the definition, context as a finite resource, context rot and the core techniques.
• Simon Willison: Context engineering (June 2025) – the term’s origin (Lütke, Karpathy) and the Karpathy definition.
• LangChain (Harrison Chase): The rise of context engineering (June 2025) – context engineering as dynamic systems, and prompt engineering as a subset.
• Hugging Face: Harness, Scaffold, and the AI Agent Terms Worth Getting Right (2026) – the context-vs-harness boundary.