Appendix B: Oracle container specification

Image: pinned by digest. Built from a Dockerfile that installs the repository's dependencies from its lockfile at build time. Rebuilt when the lockfile changes, and the new digest recorded.
Network: none. Started with networking disabled.
Mounts: the task's hidden tests, read-only; the diff file, read-only, in grade mode. Nothing from the agent's working copy.
Repository: a mirror baked into the image or mounted read-only from the oracle host. Checked out at the base commit inside the container.
Environment: TZ=UTC, LC_ALL=C.UTF-8, PYTHONHASHSEED=0, SOURCE_DATE_EPOCH fixed, and whatever the repository's own test configuration needs to be deterministic.
Diff application: the diff is filtered through the allowlist and denylist first. Dropped hunks are logged with their paths. The filtered diff is applied with git apply --check before git apply; a diff that does not apply is a FAIL with the reason logged.
Hidden tests: copied into the checkout after the diff is applied, from the mount, never from the diff.
Run: the existing suite at the base commit's version, then the hidden tests. Any failure in either is a FAIL.
Timeout: a wall-clock limit per run. Exceeding it is a FAIL with the reason logged.
Output to the gate: exit code 0 or 1, and one line tests_hash=<sha256> of the sorted list of test identifiers that ran.
Output to the oracle log: everything. Test output, dropped hunks, timing, the exit reason, the image digest, the diff hash.
Baseline mode: the same run without a diff. Must return FAIL for the hidden tests and PASS for the existing suite, or the task is quarantined.

Cite this

Anderson, M. (2026). Oracle container specification. In Building a continuous delivery system for fine-tuned open-source, open-weight models trained on your organization's traces (Appendix B). macanderson.com. https://macanderson.com/research/continuous-delivery-of-fine-tuned-open-weight-models/oracle-container-specification

BibTeX

@incollection{anderson2026continuousdeliveryof,
  author    = {Anderson, Mac},
  title     = {Oracle container specification},
  booktitle = {Building a continuous delivery system for fine-tuned open-source, open-weight models trained on your organization's traces},
  year      = {2026},
  url       = {https://macanderson.com/research/continuous-delivery-of-fine-tuned-open-weight-models/oracle-container-specification}
}

Cite this

Updates by email