Context compression is worth more than a bigger model

An agent underperforms on your repository, and the available levers look like this: a bigger model, a longer window, a larger reasoning budget. All three cost money and none of them change what you are sending. The question that predicts success is not how much the model can hold. It is what fraction of what it holds bears on the task, and that is a question you can answer with instrumentation you already own.

The Shannon view of a context window#

Shannon's framework¹⁸ gives the right vocabulary. The task has an intrinsic information requirement, the minimal set of facts needed to produce the correct patch. Everything else in the window is noise relative to that task, and attention is the finite channel the signal has to pass through. Two consequences follow. First, mutual information matters more than volume: a 4k-token context where every line bears on the task carries more usable information than 100k tokens where the answer is diluted 25 to 1. Second, compression is safe when it is lossless with respect to the task, which is what deterministic slicing (parts 2 to 4) gives you and what generic truncation does not. A stack-trace parser that keeps your frames and drops framework frames is a task-aware compressor with a retention property you can state. head -c has no such property.

Four results point the same way#

Read the four together for coding agents specifically. A repository dump is long, which triggers length degradation. It is full of near-duplicate distractors, which is the most harmful noise type the fourth result identifies. Using it requires semantic rather than lexical matching, the NoLiMa condition where the drop is steepest. A raw-context design sits at the intersection of all three.

Compression as an engineering discipline#

• Retrieval precision over recall by default: start from verified anchors (part 3) and expand along graph edges (part 4) instead of dumping candidates.
• Representation changes: signatures instead of bodies (a repo map is roughly a 100 to 1 compressor¹²), schemas instead of migrations, sliced traces instead of logs.
• Budgets enforced in code: every context assembler takes a token budget and ranks content to fit it, and going over budget fails the build.
• Context utilization as a metric: what fraction of the provided tokens does the final patch depend on? In the traces this book draws on it often sits below 10%, and it is the clearest single indicator of wasted spend (part 13, and part 17 for attributing that spend to the team that caused it).

A budget the assembler cannot talk its way out of#

def assemble(task, budget=8000):
    blocks = rank(candidates(task))        # anchors first, then graph neighbours
    kept, used = [], 0
    for b in blocks:
        if used + b.tokens > budget:
            break
        kept.append(b)
        used += b.tokens
    missing = required(task) - {b.id for b in kept}
    if missing:                            # a required anchor did not fit
        raise BudgetError(task, missing, used, budget)
    return kept
# Over budget is a failure with a name, not a truncation nobody sees.

The failure matters more than the cap. When a required anchor does not fit, you want a stack trace naming the task and the anchor, because that is a retrieval bug. Silent truncation turns the same bug into a wrong patch and a confusing review.