Skip to content

feat(protocol): pre-register v0.2 — Sonnet 5.5 opens a new column; rung 4c behind xbrlkit serve, the document through xbrlkit, 7a hosted - #44

Merged
jfrench9 merged 4 commits into
mainfrom
feature/v0-1-2
Nov 9, 2026
Merged

jfrench9 merged 4 commits into
mainfrom
feature/v0-1-2

Conversation

@jfrench9

@jfrench9 jfrench9 commented Nov 8, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Pre-registers protocol v0.2 (PROTOCOL §11): the model moves to Sonnet 5.5 (claude-sonnet-5-5), which under §8 is the first-number case and opens a new column of the results table; the four forms that are ours — 5c Tavi, 7a the sec graph's tools, 7b the filing's own property graph, 7c the holon — run under the tooling as it stands (xbrlkit 0.21.4; the hosted sec repository at RoboSystems v1.22.2), one rung added (4c: the package behind xbrlkit serve's shaped tools), the document delivered to every rung of ours by xbrlkit's own search_text / read_text, rung 2 run as the anchor that reads the model's own gain on the same bytes. Same 38 questions, same k, same judge (claude-sonnet-5). Nothing carries forward by identity across a model change: the column's other rows are pre-registered with their definitions unchanged and wait on funding. §9 gains the dated erratum that defined the third versioning case (the section was pre-registered as v0.1.2 that morning and became v0.2 the same night, before any run or tag; §9 records the sequence).

The second commit carries the 2026-11-08 follow-up (the third commit is the move to v0.2 / Sonnet 5.5): the engine's 0.0 fact read found on the hosted graph that morning was fixed in RoboSystems v1.22.2 (#2008) and §11 records it as fixed and disclosed; the harness paces 7a's MCP calls under the repository's plan caps so a cap never reaches the model as a tool error; the 26 local 7b graphs are checked for the same engine read.

Changes

  • representations/xbrlkit_serve.py (new) — rung 4c and the v0.1.2 document instrument. xbrlkit.serve.build_server over the filing loaded from its package folder, in-process (read_from: file, no network); tool definitions from the server's list_tools, every call through call_tool. SERVE_TOOLS (nine given), WITHHELD (ten, each with its reason), beside() for the "+doc" controls; one loaded filing per package shared across rungs.
  • representations/mcp.py — Pacer: a tool call goes out only when fewer than 20 went out in the trailing minute and fewer than 450 in the current UTC clock hour (the sec repository's advanced plan allows 25 and 500 and answers over the cap with HTTP 429, which the MCP client hands the model as a tool error). McpClient paces every call; list_tools is not counted by the server and not paced. cli.py — the run record carries mcp_pace.
  • ladder.py — Rung.XBRLKIT_SERVE = "4c" and its spec; V0_2_RUNGS in run order (which is also the pre-registered cut order); parse_rungs("v0.2").
  • prompts.py — the 4c source paragraph (describe_filing as the first step, parallel to 7a's); 7a's paragraph now names the five whole-document tools in its second step, first step and skeleton unchanged.
  • cli.py — the 4c context builder; --document xbrlkit|harness (default xbrlkit; harness keeps v0.1.1's instrument for replication); the run record carries document_instrument, xbrlkit, mcp_url and mcp_pace.
  • bin/check_readable_text.py (just check-readable) — the §11 gate: xbrlkit's readable text against rung 2's plain text on every filing, alphanumeric ratio ±3% and 24 probes each way matched with whitespace removed. bin/artifact_hashes.py — SHA-256 of every artifact a row of the ladder reads (keys held / re_materialized), so the open rows of the Sonnet 5.5 column read the same bytes when funded. bin/check_lbug_doubles.py (just check-lbug-doubles) — the 7b engine check: every consolidated numeric fact of every 7b graph read through two filtered projections against its text value; a 0.0 against non-zero text is the LadybugDB 0.22 defect.
  • results/v0.2-sonnet-5-5/ — text-layer-gate.json (26/26), artifact-hashes.json, lbug-double-reads.json (26/26, 16,649 facts, 0 misreads, 0 other disagreements); un-ignored in .gitignore.
  • prices.json — the claude-sonnet-5-5 row: $2 / $10 per MTok, cache write $2.50, cache read $0.10 (0.05× base on this model), verified on the provider's price page 2026-11-08; Sonnet 5's row unchanged.
  • PROTOCOL.md — §9 erratum (2026-11-08, with the v0.1.2 → v0.2 sequence), §11 pre-registration as v0.2: what changed including the model (same prices, same tokenizer, thinking on by default on both models with nothing sent, recalibrated effort so token volume is measured not assumed, preserved thinking replayed unchanged, the refusal stop reason), the lineup as the Sonnet 5.5 column with its open rows, rung 2 as the anchor, 4c's definition and withheld tools (and §9's 2026-09-06 promise of "shaped tools over the loaded XBRL model" met with the document on), 7a on the hosted repository as a disclosed lock change (the lock at v1.22.2; the engine defect found, fixed and disclosed, with what stays exposed — raw read-graph-cypher on numeric_value; the plan caps and the pace; the nightly rebuild window, 7a in one sitting outside it, the server version recorded at its start and end), the instrument and its gate result, locks (Model, Prices, Judge, 7a and LadybugDB rows), cost ($118–130: Sonnet 5's per-rung figures scaled row by row by the two shakedowns' per-question ratio; cut order stated), expectations with (f) as the model-delta anchor, the two shakedown records (the Sonnet 5 harness shakedown and the Sonnet 5.5 model shakedown), the cost table re-scaled from them ($118–130). README.md — the v0.1.2 status line.
  • pyproject.toml / uv.lock — xbrlkit[lpg,mcp]>=0.21.4 (identical package code to 0.21.3). Tests: tests/test_xbrlkit_serve.py (the registration test fails on any tool the library adds that is placed in neither set; a live load of 3M's package when the corpus is present); test_controls.py pins the control count; tests/test_mcp_pace.py (the minute and hour windows, the pace under the caps, every call paced).

Protocol impact

PROTOCOL — a new first-number protocol version, pre-registered before its run: the model lock moves to Sonnet 5.5 and opens a new column; nothing carries forward. It is also RUNG FIDELITY for the rungs it runs: 7a's tool list grew by five (the lock moved to the hosted repository at v1.22.2); 7b's and 7c's artifacts were re-materialized under xbrlkit 0.21.4 (the lossless holon, the .lbdb projection); the "+doc" controls' document tools are now xbrlkit's. The pace is HARNESS: it changes when a call goes out, never what the model is handed; it adds wall clock to the 7a row, stated in §11. The token table is untouched on purpose: questions/filing-token-counts.json is one of the manifest's four hashed files and the manifest is verified unchanged; the re-materialized holon's size goes in the results README. No question or gold changes.

Testing

just test-all green (84 passed, ruff, basedpyright). All 26 corpus filings re-materialized for 7b/7c under 0.21.4; the document gate passed 26/26 (results/v0.1.2-sonnet-5/text-layer-gate.json); the 7b engine check passed 26/26 under ladybug 0.22.1 (results/v0.1.2-sonnet-5/lbug-double-reads.json). The two hosted reads that served 0.0 on 2026-11-08 re-read correct after the v1.22.2 deploy (Realty Income FY2025 us-gaap:Assets 72,795,612,000; Microsoft FY2025 us-gaap:Goodwill 119,509,000,000). Shakedown on the six template questions (3M, 0000066740-25-000006), k = 1, the eight tool rungs, Sonnet 5: results/20261108-161744-shakedown-v0-1-2 (local, never published) — 48/48 on end_turn, $5.25, no harness change needed; recorded in §11. Sonnet 5.5 shakedown, same six questions and eight rungs: results/20261108-193946-shakedown-v0-2 (local, never published) — 48/48 on end_turn, $3.24, 9 tool errors (the model's and the engines', each corrected next turn), no thinking blocks returned by either model, answers agreeing across rungs on all six; recorded in §11 with the per-rung cost ratios.

Next (maintainer): merge, tag protocol-v0.2 on main, then the run (filing-ladder run --rungs v0.2 --k 3 --provider anthropic --model claude-sonnet-5-5 --label v0.2), with 7a outside the hosted repository's nightly rebuild window (about 04:30–05:40 UTC, Tuesday to Saturday) and the key's sec plan confirmed advanced on the day (it renews 2026-11-11 19:10 UTC).

…he document through xbrlkit, 7a hosted

PROTOCOL §11 pre-registers the run: the four forms that are ours (5c, 7a, 7b, 7c)
re-run under the locks as they stand, rung 4c added, the document on every rung
of ours delivered by xbrlkit's own search_text / read_text, rung 2 re-run as the
drift control; every other row carries forward by identity with its hashes. §9
gains the dated erratum that defines this third versioning case.

Rung 4c is `xbrlkit serve` built in-process (`xbrlkit.serve.build_server` over
the filing loaded from its package folder, read_from=file, no network): tool
definitions from the server's list_tools, every call through call_tool. Nine
tools given, ten withheld each with a reason (WITHHELD), and a test that fails
on any tool the library registers that is placed in neither set.

The "+doc" controls take the same loaded filing's two text tools
(`--document xbrlkit`, the default); `--document harness` keeps v0.1.1's.
7a's source paragraph names the five whole-document tools in its second step;
the run record now carries the document instrument, the xbrlkit version and
the MCP URL.

xbrlkit pinned at 0.21.4 (identical package code to 0.21.3). All 26 holons and
property graphs re-materialized under it. `bin/check_readable_text.py`
(`just check-readable`) is the §11 gate — 26/26, published in
results/v0.1.2-sonnet-5/text-layer-gate.json — and
`bin/carried_forward_hashes.py` writes the carried-forward hashes beside it.
questions/manifest.json is unchanged.
…; §11 records the fixed engine read

The `sec` repository counts every MCP tool call against the subscriber's plan
(advanced: 25 a minute, 500 a UTC clock hour) and answers over the cap with
HTTP 429, which the MCP client hands the model as a tool error. McpClient now
waits so that fewer than 20 calls go out in the trailing minute and fewer than
450 in the current hour; the pace is written into the run's config.

PROTOCOL §11: the 0.0 fact read found on the hosted graph on 2026-11-08 is
fixed in RoboSystems v1.22.2 (#2008, the number taken from the fact's text in
the two shaped reads) and re-verified on the hosted repository; raw Cypher on
numeric_value stays exposed and is disclosed; the 7a lock moves to v1.22.2 and
names the plan, the pace and the nightly rebuild window. The 26 local 7b graphs
run the same engine, so bin/check_lbug_doubles.py reads every consolidated
numeric fact through two filtered projections against its text: 16,649 facts,
0 misreads (results/v0.1.2-sonnet-5/lbug-double-reads.json).
…s with the nine funded rows

§8's first-number case: the model lock changed, so nothing carries forward by
identity and the column's other rows are open, pre-registered with their
definitions unchanged, waiting on funding. §11 is retitled v0.2 (§9 records that
it was pre-registered as v0.1.2 the same morning and never ran); rung 2 becomes
the anchor that reads the model's own gain on the same bytes; the judge stays
claude-sonnet-5; the cost table is re-scaled from the Sonnet 5.5 shakedown,
whose slot in §11 is marked and must be filled before the tag.

prices.json gains the claude-sonnet-5-5 row: $2 / $10, cache write $2.50, cache
read $0.10 (0.05x base on this model), verified on the provider's price page.
parse_rungs("v0.2") and V0_2_RUNGS; results/v0.2-sonnet-5-5/ with
artifact-hashes.json (bin/artifact_hashes.py, formerly the carried-forward
hashes, keys held / re_materialized) beside the gate and engine-check results.
@jfrench9 jfrench9 changed the title feat(protocol): pre-register v0.1.2 — rung 4c behind xbrlkit serve, the document through xbrlkit, 7a hosted feat(protocol): pre-register v0.2 — Sonnet 5.5 opens a new column; rung 4c behind xbrlkit serve, the document through xbrlkit, 7a hosted Nov 9, 2026
…he cost table from it

48 of 48 on end_turn, $3.24 against the Sonnet 5 shakedown's $5.25 on the same
records; fewer turns, calls, errors and tokens; no thinking block returned by
either model with no thinking parameter sent. The cost table is Sonnet 5's
per-rung figures scaled row by row by the two shakedowns' per-question ratio,
rung 2 held as an upper bound: $118–130 for the nine rows plus judge.
@jfrench9
jfrench9 merged commit 4d22e69 into main Nov 9, 2026
2 checks passed
@jfrench9
jfrench9 deleted the feature/v0-1-2 branch November 9, 2026 01:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant