Stop making agents read your entire logs

Open a run where your agent debugged a failing test and you will find the same four steps every time: cat the log file, grep for "error", tail -n 200, paste the result into context. The most expensive component in the stack is now doing the job of a parser, and doing it probabilistically.

Signal to noise is a system property#

A production stack trace from a Python web request can run hundreds of frames, and most of them are framework internals: WSGI plumbing, middleware, object-relational mapper (ORM) dispatch. The frames that matter for the bug are usually the handful that touch your code. An irrelevant frame is not neutral padding. It costs tokens on the way in, and it competes for attention once it is there. Chroma's "Context Rot" study evaluated 18 frontier models and found that performance degrades as input length grows, even on simple tasks, and that it degrades faster when the context holds distractors that are related to the target without being the target.⁷ A raw log is a distractor-dense document by construction: hundreds of lines that look like the one line you need.

Retrieval research reports the same shape. Cuconasu et al. ("The Power of Noise," SIGIR 2024) found that adding semantically related but non-answer-bearing documents to a model's context degrades accuracy, and that related noise hurts more than unrelated noise, because the model cannot cheaply dismiss it.⁹ Framework frames in a stack trace are that kind of high-similarity distractor.

Execution slicing: preprocess, do not prompt#

The deterministic alternative treats program output as structured data and slices it before inference:

• Stack trace reduction: parse the trace, keep frames inside your package roots, collapse framework frames to one-line markers, keep the exception type, the message, and the innermost user frame's local variables.
• Dynamic trace extraction: run the failing test under a tracer (sys.settrace, coverage.py, eBPF, OpenTelemetry) and extract the execution path that actually ran, with the values that flowed through it.
• Log windowing: anchor on the failure timestamp or request id and take a bounded, correlated window instead of the file.

# Instead of handing the model a shell and hoping:
#   grep -rn "error" logs/ | tail -500   ← the model pays for every candidate line

result = run_code_with_trace(          # deterministic, runs in milliseconds
    cmd="pytest tests/test_billing.py::test_refund -x",
    package_roots=["app/"],           # slice to code you own
    capture=["exception", "locals", "executed_lines", "sql"],
    max_tokens=1200,                   # hard budget, enforced by code
)
context = result.to_prompt_block()     # 1.2k tokens of signal

The difference is architectural, not cosmetic. grep | cat | tail makes the model the filter. You pay tokens for every candidate line, and the filtering step itself is probabilistic. run_code_with_trace() makes code the filter. Filtering costs no tokens, returns the same slice on the same input, and you can test it like any other function. Bash is a fine escape hatch. It is a poor primary retrieval engine, because each round trip is another model call that carries the whole conversation history with it.