[jfrench9]

Summary

Publishes protocol v0.2.1's two rows — 4c.1 and 7a.1, each its v0.2 base with one friction removed — run 2026-11-09 18:05–18:46 UTC on claude-sonnet-5-5 under the protocol-v0.2.1 tag, judged by claude-sonnet-5, $17.61 against §12's $15–18. Both rows landed where their bases did on accuracy, every paired change inside one standard error. 7a.1 did what §12 said: zero schema or example fetches in 114 records, 14% fewer tool calls, cost per correct answer down on both tiers ($0.054 / $0.158 against 7a's $0.062 / $0.172). 4c.1 did half: refusals 44 → 4, but turns per record unchanged — the model spent the recovered turns reading — and the schemas ride along as cached tokens every turn, so the open-source row cost more per correct lookup ($0.076 against $0.061) for one more correct answer in sixty. Given both argument names, the model used elements 22 times and concepts 22 times. 7a.1 read Airbnb's other 10-Ks on vals-17 and vals-22 again and the §11 rule scored those six answers confident-wrong, as it had for 7a. Expectations: (a), (c), (d), (e) held; (b) contradicted; (f) held for 7a.1 and contradicted for 4c.1 on T1. The §9 erratum's reading that the schema defect had worked against 4c's cost was wrong, and the README says so.

Changes

• results/v0.2.1-sonnet-5-5/ — README.md (the run, what the rows removed, the result, accuracy and cost beside each base, the paired changes, the cross-filing rule applied, tool behaviour with the 4c cost anatomy by tier, the six expectations scored, files); judgments.jsonl (228); form-scoring.json (the accession scan on 7a.1: eight records with a foreign accession, two only in search listings, six re-scored); run.json, summary.json, summary.md; status-start.json / status-end.json (RoboSystems 1.22.2 healthy at both ends); transcripts.jsonl.gz (SHA-256 b4877cb6dce4811892c7cf720fa49f404f4899c387121d7c03737827c450c948).
• README.md — the status paragraph: v0.2.1 has run, with the one-paragraph result.

Nothing under results/v0.2-sonnet-5-5/ changes; the v0.2 rows carry forward by identity.

Protocol impact

HARNESS — no code change; this PR adds a published results directory under the pre-registered §12 and reports its expectations as scored. The one correction of protocol text is in the results README, not in PROTOCOL.md: §9's 2026-11-09 erratum said the served-schema defect had worked against 4c's cost; the row with the schemas cost more, and the README records that the erratum's direction was wrong on cost.

Testing

just test-all green. Run directory results/20261109-120507-v0.2.1 (local; its records are the published transcripts.jsonl.gz), judged in place; shakedown results/20261109-120218-shakedown-v0-2-1 (12 records, $0.37, never scored). The run was pulsed every 20 minutes: 0 non-200 API responses, 0 429s, the pacer never engaged.

Added in the second commit — every published row in one table

Joey asked for one table of all the results with the comparative metrics; there was none — each run's README compares within its run and results/README.md still indexed only v0.1 and v0.1.1. Now:

• bin/results_table.py (just results-table) reads every published results/v*/ run — summary.json, run.json, and form-scoring.json where the cross-filing rule applied — and rewrites a marked block in results/README.md: every row of every version (version, model, row, form, T1 and T2 accuracy, confident-wrong, cost per correct, cost per question; form-scored cells marked ‡), ranked within each version by cost per correct lookup; and the rows that recur across versions, the cost curve. 31 rows across 4 runs today.
• results/README.md — the run index gains v0.2 and v0.2.1; the generated block follows.
• results/v0.2.1-sonnet-5-5/form-scoring.json — its 7a.1_T2 block takes the shape v0.2's 7a_T2 has (judge / form_scored sub-objects) so one reader serves both; the figures are unchanged.
• README.md — one pointer sentence in Status.

Protocol impact unchanged: HARNESS. No score is computed here that a run's own files do not already carry.