Move to a newer model
0.10.1 → 0.11.0| Status | inconclusive |
| Procedure touched | No procedure change |
| Tested against | 150-field standard set · 5 of 5 runs, strict |
| Evidence | 93.8% ±1.9 against 92.4% ±2 baseline |
| Spent | $10.60 · 19m |
| Owner | s.mendo |
What changed in the procedure
before and after, by stepWhat it did to the queue
the reviewer’s side of the loopNo measurable change to what stops. The lesson is about evidence rather than the model: a 1.4-point move needs roughly 20 runs to prove at this spread, and five will never do it.
Thread
verdict computed as inconclusiveA newer model is available. Raise it and the reasoning effort, change nothing about the procedure, and grade against the standard set before we touch any of the steps.
93.8% ±1.9 against a 92.4% ±2.0 baseline. The intervals overlap, so the 1.4-point gain is not separable at five runs. Cost per file rose nineteen cents from the higher effort.
worth running it at twelve to find out?
Probably not on its own. At this spread a 1.4-point move needs roughly 20 runs to separate, about $42. The same money spent on the step-4 change bought four points at twelve runs.
Model upgrades are cheap to adopt and expensive to prove. The bundle carries the newer model either way — the honest record is that this attempt did not demonstrate the gain.