Pine adapter: per-bar and per-fill walks read only the cohorts that hold an opening or live units, and the roster pass only the rosters with members (#452)

[letiep97]

Refs #452.

What

PineExecutionAdapter keeps a cohort in cohorts_by_id_ / cohort_order_ for every entry id the run has used, and the core keeps a CohortRoster for each of their handles; neither is ever pruned, and neither should be (origin history and the state hash read them). Several walks that only read opened and live_units_by_origin went over all of them: the open-entry count the scheduler publishes every bar (source_entry_slot_count), the two walks in erase_retired_rows, the bar-open roster pass in release_closed_cohort_origins, and some twenty more at fills and placements. A script that gives each position an id of its own therefore paid the square of its trade count. This change adds held_cohort_ids_, the ids whose cohort holds an opening or live units, kept in cohort_order_ order (which is handle order) as opened and live_units_by_origin change, and points those walks at it. release_closed_cohort_origins keeps a memo of the rosters that hold members, fed by the core's append-only cohort_receipts() (only an applied add puts a member in a roster), and its four loops read those rosters only. Both are derived, not state: held_cohort_ids_ is waived in broker_state_hash_waivers.txt like the other indexes, and the roster memo lives in retired_row_scratch_, whose waiver now says so. In every non-Release read, held_cohort_ids() asserts the index equals the cohorts a full walk finds, in the same order, and source_entry_slot_count and the roster pass assert their totals against the walks they replace. Trade lists and equity are unchanged.

Not covered: a fill that reverses the position still collects every cohort id the run has used and calls cancel_exit_orders_for_full_close for each (on_applied, the flipped_position branch). For an id that matches nothing, that call still sweeps bracket_families_, so narrowing the set is not obviously equivalent; I left it for you to judge. On the synthetic below, the reversing variant goes from 219.6 s to 23.6 s, against about 1.4 s with fixed ids.

The anchor moves are mechanical: every pine_adapter.cpp / pine_adapter.hpp path:line anchor in docs and test comments was remapped to the line its symbol moved to (check_doc_anchors.py: 1742/1742). scripts/corpus_parity_mutation_battery.tsv is left alone: its lines already name an older tree at base.

Evidence

New row uniqueids in tests/test_adapter_live_state_scaling.cpp: an opening every 20 bars under an id of its own, closed 10 bars later, no recording. Failing before, this TU against base 0212cabe (macOS 15.8 arm64, Apple M1 Pro, Apple clang 17, Release):

FAIL .../tests/test_adapter_live_state_scaling.cpp:476 ratio < kShapeBound
an opening every 20 bars under an id of its own, closed 10 bars later (no recording): 32000 bars 0.3703s (1600 trades, 0 rows retained, 0 roster members), 128000 bars 8.4500s (6400 trades, 0 rows retained, 0 roster members), ratio 22.82 (bound 5.0), rows bound 64, roster bound 8

After:

an opening every 20 bars under an id of its own, closed 10 bars later (no recording): 128000 bars 0.1011s (6400 trades, 0 rows retained, 0 roster members), 512000 bars 0.4149s (25600 trades, 0 rows retained, 0 roster members), ratio 4.11 (bound 5.0), rows bound 64, roster bound 8
test_adapter_live_state_scaling: ok

python3 scripts/ci_verify.py release --build-dir build-ci-release --jobs 8 (full population, no label excluded), build-ci-release/ci-summary.json:

'profile': 'release', 'status': 'passed', 'exitCode': 0, 'ctestRegistered': 960, 'ctestSelected': 960, 'minTests': 603
stages: 31 passed, 8 skipped (source-guard-native-versions, abi-base, abi-prior, abi-v13, abi-v14, abi-v15-frozen, abi-v16-frozen, abi-v18-frozen: SKIP private fixtures not present)
100% tests passed out of 960

ci_verify.py kernel: not run. Its configure stops at Could NOT find CURL: Found unsuitable version "8.7.1", but required is at least "8.14.1" (the macOS SDK's libcurl, for the native runner). This change only compiles with PINEFORGE_BUILD_SOURCE_LAYER, which that profile turns off.

ctest, Debug build with the new asserts on, every target, -j8 -E '^test_ci_verify$':

99% tests passed, 1 tests failed out of 940
	 35 - test_run_strategy_requests (Failed)

That row expects a RecursionError that Homebrew Python 3.14.6's json no longer raises; it fails the same way on base 0212cabe and passes under Python 3.14.4, which ci_verify used.

./scripts/check_corpus_parity.sh --subset:

corpus_trades_identity: 54 of 312 probes (subset scripts/corpus_parity_subset.txt) at corpus e99bbd9db257740d9c77db37a8567616627130bf — baseline match=54, moved=0, unrecorded=0, missing=0, schema violations=0
[corpus-parity] TradingView parity holds on the 54-probe subset at corpus e99bbd9db257740d9c77db37a8567616627130bf

./scripts/check_corpus_parity.sh (full sweep):

corpus_trades_identity: 312 probes at corpus e99bbd9db257740d9c77db37a8567616627130bf — baseline match=312, moved=0, unrecorded=0, missing=0, schema violations=0
Verified 312 strategies — excellent=311, strong=0, moderate=0, weak=0, minimal=0, anomaly=1, engine_only=0, missing=0
[corpus-parity] TradingView parity holds at corpus e99bbd9db257740d9c77db37a8567616627130bf

python3 scripts/ci_preflight.py: 48 passed, 4 failed, 1 skipped. actionlint-version, shellcheck-version and workflow-lint fail because neither tool is installed here; run-json-harness-tests has 9 failures from Python 3.14's argparse prog (usage: python3 -m pytest ...), identical on base.

The script from #452 and three variants (fixed ids; reversing with strategy.entry alone instead of strategy.close_all() first), Binance spot BTCUSDT 5m, 2020-01-01 to 2026-09-07, 702,505 bars, 13,500 trades each, one run_backtest_full per run, strategy built -O2 against each tree's Release libpineforge.a. Trades and equity hashed field by field; every run on both trees printed trades=13500 net=-286.760100 trades_sha=d6a9d963a7427d46 equity_sha=658408382d422c20 equity_len=702505.

Script	Base 0212cabe	This branch
unique ids, close_all first (#452)	232.09 s	1.72 s
unique ids, reversing	219.58 s	23.56 s
fixed ids, close_all first (median of 10)	1.02 s	1.00 s
fixed ids, reversing (median of 4)	1.36 s	1.40 s

Out of scope: with recording on, hash_state still walks every cohort, and each id used keeps about 5 KB (about 271 MB RSS at 51k ids).

🤖 Generated with Claude Code