Context Engineering

A model's context window is finite and every token in it costs money and (crucially) attention. Context engineering is the discipline of deciding, for a given request, exactly *what* goes into that limited space — which documents, which conversation turns, which tool results, in what order — so the model has what it needs and nothing that dilutes or crowds out the signal.

• Context window — the maximum number of tokens (input + output combined, model-dependent) the model can attend to at once.
• RAG (retrieval-augmented generation) — fetching only the most relevant documents from a large corpus and inserting them into context, instead of trying to fit everything.
• Context rot / lost-in-the-middle — the empirical finding that models are less reliable at using information buried in the middle of a long context than at the start or end.
• Memory / context compression — summarizing or pruning older conversation turns so a long-running session doesn't blow the context budget.

This became its own discipline once people started building things with tens of thousands of tokens of context — long documents, long conversations, many tool calls — and discovered that simply concatenating everything relevant into the prompt stopped working well, both for cost and for quality.

• Good context engineering lets small, cheap models perform like expensive ones on knowledge-heavy tasks, because the hard part (finding the right information) is done outside the model.
• It's the backbone of virtually every production LLM system with private data: customer support bots, code assistants indexing a codebase, research assistants over a document set.
• Its failure modes are subtle: retrieving the wrong chunk, retrieving too much, or losing earlier facts to summarization all degrade output *without* an obvious error message.

Think of context engineering as a pipeline: decide what's available (a corpus, a history, tool results), select a relevant subset (retrieval, filtering), and arrange it (ordering, formatting, compression) before it ever reaches the model. Later layers dig into how each stage actually works, mechanically and mathematically.