Repository navigation
feat(protocol): pre-register v0.2 — Sonnet 5.5 opens a new column; rung 4c behind xbrlkit serve, the document through xbrlkit, 7a hosted - #44
Merged
Conversation
…he document through xbrlkit, 7a hosted PROTOCOL §11 pre-registers the run: the four forms that are ours (5c, 7a, 7b, 7c) re-run under the locks as they stand, rung 4c added, the document on every rung of ours delivered by xbrlkit's own search_text / read_text, rung 2 re-run as the drift control; every other row carries forward by identity with its hashes. §9 gains the dated erratum that defines this third versioning case. Rung 4c is `xbrlkit serve` built in-process (`xbrlkit.serve.build_server` over the filing loaded from its package folder, read_from=file, no network): tool definitions from the server's list_tools, every call through call_tool. Nine tools given, ten withheld each with a reason (WITHHELD), and a test that fails on any tool the library registers that is placed in neither set. The "+doc" controls take the same loaded filing's two text tools (`--document xbrlkit`, the default); `--document harness` keeps v0.1.1's. 7a's source paragraph names the five whole-document tools in its second step; the run record now carries the document instrument, the xbrlkit version and the MCP URL. xbrlkit pinned at 0.21.4 (identical package code to 0.21.3). All 26 holons and property graphs re-materialized under it. `bin/check_readable_text.py` (`just check-readable`) is the §11 gate — 26/26, published in results/v0.1.2-sonnet-5/text-layer-gate.json — and `bin/carried_forward_hashes.py` writes the carried-forward hashes beside it. questions/manifest.json is unchanged.
…; §11 records the fixed engine read The `sec` repository counts every MCP tool call against the subscriber's plan (advanced: 25 a minute, 500 a UTC clock hour) and answers over the cap with HTTP 429, which the MCP client hands the model as a tool error. McpClient now waits so that fewer than 20 calls go out in the trailing minute and fewer than 450 in the current hour; the pace is written into the run's config. PROTOCOL §11: the 0.0 fact read found on the hosted graph on 2026-11-08 is fixed in RoboSystems v1.22.2 (#2008, the number taken from the fact's text in the two shaped reads) and re-verified on the hosted repository; raw Cypher on numeric_value stays exposed and is disclosed; the 7a lock moves to v1.22.2 and names the plan, the pace and the nightly rebuild window. The 26 local 7b graphs run the same engine, so bin/check_lbug_doubles.py reads every consolidated numeric fact through two filtered projections against its text: 16,649 facts, 0 misreads (results/v0.1.2-sonnet-5/lbug-double-reads.json).
…s with the nine funded rows
§8's first-number case: the model lock changed, so nothing carries forward by
identity and the column's other rows are open, pre-registered with their
definitions unchanged, waiting on funding. §11 is retitled v0.2 (§9 records that
it was pre-registered as v0.1.2 the same morning and never ran); rung 2 becomes
the anchor that reads the model's own gain on the same bytes; the judge stays
claude-sonnet-5; the cost table is re-scaled from the Sonnet 5.5 shakedown,
whose slot in §11 is marked and must be filled before the tag.
prices.json gains the claude-sonnet-5-5 row: $2 / $10, cache write $2.50, cache
read $0.10 (0.05x base on this model), verified on the provider's price page.
parse_rungs("v0.2") and V0_2_RUNGS; results/v0.2-sonnet-5-5/ with
artifact-hashes.json (bin/artifact_hashes.py, formerly the carried-forward
hashes, keys held / re_materialized) beside the gate and engine-check results.
…he cost table from it 48 of 48 on end_turn, $3.24 against the Sonnet 5 shakedown's $5.25 on the same records; fewer turns, calls, errors and tokens; no thinking block returned by either model with no thinking parameter sent. The cost table is Sonnet 5's per-rung figures scaled row by row by the two shakedowns' per-question ratio, rung 2 held as an upper bound: $118–130 for the nine rows plus judge.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Pre-registers protocol v0.2 (PROTOCOL §11): the model moves to Sonnet 5.5 (
claude-sonnet-5-5), which under §8 is the first-number case and opens a new column of the results table; the four forms that are ours — 5c Tavi, 7a thesecgraph's tools, 7b the filing's own property graph, 7c the holon — run under the tooling as it stands (xbrlkit 0.21.4; the hostedsecrepository at RoboSystems v1.22.2), one rung added (4c: the package behindxbrlkit serve's shaped tools), the document delivered to every rung of ours by xbrlkit's ownsearch_text/read_text, rung 2 run as the anchor that reads the model's own gain on the same bytes. Same 38 questions, same k, same judge (claude-sonnet-5). Nothing carries forward by identity across a model change: the column's other rows are pre-registered with their definitions unchanged and wait on funding. §9 gains the dated erratum that defined the third versioning case (the section was pre-registered as v0.1.2 that morning and became v0.2 the same night, before any run or tag; §9 records the sequence).The second commit carries the 2026-11-08 follow-up (the third commit is the move to v0.2 / Sonnet 5.5): the engine's 0.0 fact read found on the hosted graph that morning was fixed in RoboSystems v1.22.2 (#2008) and §11 records it as fixed and disclosed; the harness paces 7a's MCP calls under the repository's plan caps so a cap never reaches the model as a tool error; the 26 local 7b graphs are checked for the same engine read.
Changes
representations/xbrlkit_serve.py(new) — rung 4c and the v0.1.2 document instrument.xbrlkit.serve.build_serverover the filing loaded from its package folder, in-process (read_from: file, no network); tool definitions from the server'slist_tools, every call throughcall_tool.SERVE_TOOLS(nine given),WITHHELD(ten, each with its reason),beside()for the "+doc" controls; one loaded filing per package shared across rungs.representations/mcp.py—Pacer: a tool call goes out only when fewer than 20 went out in the trailing minute and fewer than 450 in the current UTC clock hour (thesecrepository's advanced plan allows 25 and 500 and answers over the cap with HTTP 429, which the MCP client hands the model as a tool error).McpClientpaces everycall;list_toolsis not counted by the server and not paced.cli.py— the run record carriesmcp_pace.ladder.py—Rung.XBRLKIT_SERVE = "4c"and its spec;V0_2_RUNGSin run order (which is also the pre-registered cut order);parse_rungs("v0.2").prompts.py— the 4c source paragraph (describe_filing as the first step, parallel to 7a's); 7a's paragraph now names the five whole-document tools in its second step, first step and skeleton unchanged.cli.py— the 4c context builder;--document xbrlkit|harness(default xbrlkit;harnesskeeps v0.1.1's instrument for replication); the run record carriesdocument_instrument,xbrlkit,mcp_urlandmcp_pace.bin/check_readable_text.py(just check-readable) — the §11 gate: xbrlkit's readable text against rung 2's plain text on every filing, alphanumeric ratio ±3% and 24 probes each way matched with whitespace removed.bin/artifact_hashes.py— SHA-256 of every artifact a row of the ladder reads (keysheld/re_materialized), so the open rows of the Sonnet 5.5 column read the same bytes when funded.bin/check_lbug_doubles.py(just check-lbug-doubles) — the 7b engine check: every consolidated numeric fact of every 7b graph read through two filtered projections against its textvalue; a 0.0 against non-zero text is the LadybugDB 0.22 defect.results/v0.2-sonnet-5-5/—text-layer-gate.json(26/26),artifact-hashes.json,lbug-double-reads.json(26/26, 16,649 facts, 0 misreads, 0 other disagreements); un-ignored in.gitignore.prices.json— theclaude-sonnet-5-5row: $2 / $10 per MTok, cache write $2.50, cache read $0.10 (0.05× base on this model), verified on the provider's price page 2026-11-08; Sonnet 5's row unchanged.PROTOCOL.md— §9 erratum (2026-11-08, with the v0.1.2 → v0.2 sequence), §11 pre-registration as v0.2: what changed including the model (same prices, same tokenizer, thinking on by default on both models with nothing sent, recalibrated effort so token volume is measured not assumed, preserved thinking replayed unchanged, therefusalstop reason), the lineup as the Sonnet 5.5 column with its open rows, rung 2 as the anchor, 4c's definition and withheld tools (and §9's 2026-09-06 promise of "shaped tools over the loaded XBRL model" met with the document on), 7a on the hosted repository as a disclosed lock change (the lock at v1.22.2; the engine defect found, fixed and disclosed, with what stays exposed — rawread-graph-cypheronnumeric_value; the plan caps and the pace; the nightly rebuild window, 7a in one sitting outside it, the server version recorded at its start and end), the instrument and its gate result, locks (Model, Prices, Judge, 7a and LadybugDB rows), cost ($118–130: Sonnet 5's per-rung figures scaled row by row by the two shakedowns' per-question ratio; cut order stated), expectations with (f) as the model-delta anchor, the two shakedown records (the Sonnet 5 harness shakedown and the Sonnet 5.5 model shakedown), the cost table re-scaled from them ($118–130).README.md— the v0.1.2 status line.pyproject.toml/uv.lock—xbrlkit[lpg,mcp]>=0.21.4(identical package code to 0.21.3). Tests:tests/test_xbrlkit_serve.py(the registration test fails on any tool the library adds that is placed in neither set; a live load of 3M's package when the corpus is present);test_controls.pypins the control count;tests/test_mcp_pace.py(the minute and hour windows, the pace under the caps, everycallpaced).Protocol impact
PROTOCOL — a new first-number protocol version, pre-registered before its run: the model lock moves to Sonnet 5.5 and opens a new column; nothing carries forward. It is also RUNG FIDELITY for the rungs it runs: 7a's tool list grew by five (the lock moved to the hosted repository at v1.22.2); 7b's and 7c's artifacts were re-materialized under xbrlkit 0.21.4 (the lossless holon, the
.lbdbprojection); the "+doc" controls' document tools are now xbrlkit's. The pace is HARNESS: it changes when a call goes out, never what the model is handed; it adds wall clock to the 7a row, stated in §11. The token table is untouched on purpose:questions/filing-token-counts.jsonis one of the manifest's four hashed files and the manifest is verified unchanged; the re-materialized holon's size goes in the results README. No question or gold changes.Testing
just test-allgreen (84 passed, ruff, basedpyright). All 26 corpus filings re-materialized for 7b/7c under 0.21.4; the document gate passed 26/26 (results/v0.1.2-sonnet-5/text-layer-gate.json); the 7b engine check passed 26/26 under ladybug 0.22.1 (results/v0.1.2-sonnet-5/lbug-double-reads.json). The two hosted reads that served 0.0 on 2026-11-08 re-read correct after the v1.22.2 deploy (Realty Income FY2025us-gaap:Assets72,795,612,000; Microsoft FY2025us-gaap:Goodwill119,509,000,000). Shakedown on the six template questions (3M,0000066740-25-000006), k = 1, the eight tool rungs, Sonnet 5:results/20261108-161744-shakedown-v0-1-2(local, never published) — 48/48 onend_turn, $5.25, no harness change needed; recorded in §11. Sonnet 5.5 shakedown, same six questions and eight rungs:results/20261108-193946-shakedown-v0-2(local, never published) — 48/48 onend_turn, $3.24, 9 tool errors (the model's and the engines', each corrected next turn), no thinking blocks returned by either model, answers agreeing across rungs on all six; recorded in §11 with the per-rung cost ratios.Next (maintainer): merge, tag
protocol-v0.2on main, then the run (filing-ladder run --rungs v0.2 --k 3 --provider anthropic --model claude-sonnet-5-5 --label v0.2), with 7a outside the hosted repository's nightly rebuild window (about 04:30–05:40 UTC, Tuesday to Saturday) and the key'ssecplan confirmed advanced on the day (it renews 2026-11-11 19:10 UTC).