Your context window is a budget, not a bucket

A year ago the craft was called prompt engineering, and it was mostly about phrasing — the magic incantation, the role-play preamble, the “think step by step.” That craft didn’t disappear; it got demoted. The discipline that replaced it at the center is context engineering: deciding what information goes into the model’s window, in what order, and what gets left out. The rename matters because it changes the mental model. A prompt is something you write. A context is something you budget.

The instinct, when a model gets a bigger window, is to treat it as a bucket: pour everything in — the whole document, the full chat history, every retrieved chunk, all the tool output — and let the model sort it out. Bigger bucket, fewer decisions. That instinct is wrong in a specific, measurable way, and knowing how it’s wrong is the whole skill.

Two things degrade as you fill the window. The first is attention to position. Models don’t read a long context evenly; information in the middle of a long input gets used less reliably than information at the very start or the very end — the effect the research community labeled “lost in the middle.” Bury the one fact the answer depends on halfway down a 100-page dump and you’ve technically “given” the model the fact while making it hard to actually use. The second is signal-to-noise. Every irrelevant token you add is another thing the model has to hold down and ignore — and, like a person, it doesn’t ignore perfectly. More context can make an answer worse, not just slower and more expensive.

So the window is a budget with three line items, and context engineering is spending it well.

Selection is the biggest lever, and it’s why retrieval beats stuffing. Instead of pasting the whole knowledge base and hoping, you fetch the few passages actually relevant to this turn and spend your tokens on those. The failure mode of stuffing isn’t just cost — it’s that you drown the answer in near-misses.

Ordering is the cheapest lever and the most ignored. If position changes how reliably information is used, then the load-bearing material — the instruction the model must follow, the fact the answer hinges on, the schema it must match — belongs at the edges, not buried in the soft middle. Put the task restatement right before the model has to act on it, not paragraphs above.

Compaction is the lever that keeps long-running things alive. A conversation or an agent loop accumulates history until it crowds out room to think. The fix is to periodically summarize the old turns into a compact state and drop the raw transcript — trade a faithful, expensive record for a lossy, cheap one, on purpose. What you keep verbatim and what you compress is a design decision, not an accident of scrollback.

There’s a familiar shape underneath all this, and it’s worth saying without overclaiming: the constraints look a lot like a working memory. A limited store, unreliable in the middle, that performs best when you hand it a few well-chosen, well-placed things rather than everything you own. You don’t help a colleague by emailing them the entire wiki; you help them by sending the one paragraph and saying where it fits. Context engineering is that, made explicit and metered.

The practical takeaway for anyone shipping on top of a model: stop asking “does it fit in the window?” and start asking “what’s the least I can put in the window to make this turn reliable?” The bigger windows didn’t remove the budget. They just raised the ceiling and made it easier to overspend without noticing.

The systems, to look up: “Lost in the Middle: How Language Models Use Long Contexts” (Liu et al., 2023, TACL) — the positional-reliability effect; the 2025-26 shift in practitioner writing from prompt engineering to context engineering. Hedge: exact effect sizes vary by model and task, and newer models mitigate some positional weakness — treat the shape as robust, the magnitudes as model-specific.