Overview
Autonomy
Share of files cleared with no humanRising through the window — still no substitute for sampled accuracy on the blind-review tab.
Cost per run
Inference cost per completed runCost drifts with the period average; the tile above is the latest day, not the chart peak.
What this doesn’t tell you: sampled accuracy is 95.1% on 70 fields — enough to catch a systematic miss, not an interval. Graded corpus is 96.4% ±1.2 (lab).
| HL-40128 | conflicting extraction | Outcome unchanged either way | 12m |
| HL-40093 | missing document | Outcome unchanged either way | 9m |
| HL-40077 | low confidence | Outcome unchanged either way | 3m |
| HL-40190 | conflicting extraction | Outcome unchanged either way | 2m |
| HL-40244 | low confidence | Outcome unchanged either way | 1m |
| HL-40211 | policy judgment | Changes the outcome | 6m |
| HL-40155 | escalation · adverse action | Changes the outcome | 4m |
Sorted by wait time within its routing class, not by a priority score — a score would need tuning and would be gamed. The third column is computed from the policy cards: most conflicts do not move the decision, and saying so is what keeps the queue moving.
| batch | bundle | state | files | autonomy | held | spent |
|---|---|---|---|---|---|---|
| batch-0903-am | 0.12.0 | running | 108 | 86% | 7 | $237 |
| batch-0902-pm | 0.11.0 | closed | 96 | 79% | — | $222 |
Opens on held — 93 cleared alone in the current batch.
| HL-40119 | 14 fields · 0 corrections | agreed | s.mendo · 12:18 |
| HL-40086 | 14 fields · 1 correction | InvestmentAccounts | j.park · 12:09 |
| HL-40044 | 14 fields · 0 corrections | agreed | j.park · 11:52 |
| HL-39988 | awaiting review | open | drawn 11:31 |
Blind to which files are sampled — the only control that catches confident errors the queue never sees.
| entry | kind | metric | runs | cost | verdict | note |
|---|---|---|---|---|---|---|
| batch-0903-am | batch | 86% autonomy | 108 | $237 | running | first batch on 0.12.0 |
| 0.12.0 | eval | 96.4% ±1.2 | 12 | $26.28 | keep | step-7 title review; checker sees obligation lines |
| 0.11.1 | eval | — | 2 | $3.10 | crash | bundle load failed, step-7 skill missing |
| 0.11.0 | eval | 93.8% ±1.9 | 5 | $10.60 | inconclusive | Opus 4.8 to Opus 5, effort xhigh |
| batch-0902-pm | batch | 79% autonomy | 96 | $222 | closed | last batch on 0.11.0 |
| 0.10.1 | eval | 88.9% ±2.4 | 5 | $11.40 | discard | dropped page citations from document-reader |
| 0.10.0 | eval | 93.1% ±2.1 | 5 | $11.55 | inconclusive | split step-4 into obligations and qualifying payment |
| 0.9.2 | eval | 92.4% ±2.0 | 5 | $12.25 | baseline | HELOC File Review v1, Opus 4.8 |
Evals report accuracy; batches report autonomy. Append-only — crash and discard stay.
The checker only sees the step it is verifying, so it cannot cross-check reserves or the flood determination against the rest of the file.
Sixteen of twenty obligation conflicts resolved to the lender statement. Stop asking when the two sources are close.
Two open — one grading, one drafted. Draft cuts three of four conflicting extractions; INV-4 still applies.