Data volume worksheet

Fill in the first four lines from your own team. The rest is arithmetic.

A  sessions per day                   = engineers × sessions each       = ____
B  oracle share                       = fraction of sessions on tasks
                                        with a hidden test               = ____
C  flip rate per attempt              = measured in week one             = ____
D  attempts per task                  = independent sessions dispatched  = ____

E  oracle tasks per day               = A × B                            = ____
F  chance a task flips at least once  = 1 − (1 − C)^D                    = ____
G  flips per day                      = E × F                            = ____
H  token cost multiplier              = D                                = ____

Days to 500 flips   = 500 / G
Days to 2,000 flips = 2,000 / G
Days to 8,000 flips = 8,000 / G

Reference points from Chapter 12: 491 flips moved a 32-billion-parameter model from 7.0 to 20.6 percent on SWE-bench Verified; about 8,000 moved the same base model to 38.0 percent with the gain still growing.

@incollection{anderson2026continuousdeliveryof,
  author    = {Anderson, Mac},
  title     = {Data volume worksheet},
  booktitle = {Building a continuous delivery system for fine-tuned open-source, open-weight models trained on your organization's traces},
  year      = {2026},
  url       = {https://macanderson.com/research/continuous-delivery-of-fine-tuned-open-weight-models/data-volume-worksheet}
}