[mooki000]

What this is for: a script steering Link needs his position, camera direction and current game state before its next input. This moves the proposed state-file stream into the control bridge already merged in #926, so tools use one protocol rather than a second output channel. Work is tracked here; #2335 was closed to keep it in this pull request. What is measured: on Arch Linux/RTX 4090, gcc and clang pass the socket sampler and bridge checks; each rejects 38 compiled behavioral regressions. A copied-save comparison found nine standing-state fields identical over file and socket, and a watched Bool transition also appeared in changed. Not measured: Windows, AMD/Intel GPUs, rendering parity, frame-rate cost, long play sessions or real controllers.

Problem

• host/control.c already offers status and subscriptions, and docs/control.md describes it as the connection for tools.
• The first version of this PR wrote a separate file. It duplicated map/event/loading/age reporting and made consumers combine two channels.
• Reading the remaining fields directly from the socket thread would be the wrong move: the actor and manager pointers belong to the game's update. The reads must stay there, with copied results crossing to the bridge.

What changed

Files	Change
host/oot_state_tap.c	Retains the bounded, read-only guest readers in Link's update. Publishes a copied core snapshot and per-connection samples instead of opening/writing/flushing a file. Each subscription has its own watch list and changed-flag baseline.
host/control.h, host/control.c	HosControlGame callbacks copy the latest snapshot, replace/cancel a watch and consume a pending sample. status.game carries the core fields; the explicitly requested oot event carries samples with optional flags/changes. Disconnect, replacement and unsubscribe clear the old watch.
host/hos_main.c	Keeps sampling under the existing debug-tools lock. Removes the superseded three-flag snapshot callback; the existing last-known-map cache remains.
scripts/hookshot_ctl.py	Plain and JSON status show the snapshot. watch --events oot --every N --flags NAME,0xHASH --changes exposes the stream.
docs/control.md, docs/development.md	Document the fields, units, cadence, nulls, stale snapshots, limits and migration to the bridge. The four state-file switches are removed.
Tests	Carry the old sampler assertions into a bounded fake guest connected to the real socket/pipe bridge; preserve the bridge/client tests and add subscription lifecycle, payload-boundary and focused regression checks.
docs/tested-configurations.md	Records the completed comparison, the interrupted first attempt, and the visual limitation rather than claiming rendering parity.

The new core fields are t, sample frame, map, pos, facing, camera, life, max_life, rupees, prompt and music, alongside the bridge's event, loading and age. Unknown/unreadable values are null. Position is in metres, yaw in degrees, life in quarter hearts. The top-level map remains the last known map; game.map is the sampled value.

flags accepts up to 64 names or explicit hashes. Enum values retain the #<hash> representation. changes:true scans Bool/Int/Enum/UInt values: the first sample establishes a baseline, at most 32 changes are returned, and more counts the remainder. Both lookup and scanning reject tables above 2^22 slots; d847c0c's cap is retained. A pending sample is not overwritten before the bridge takes it.

Protocol 1 is retained for these additive fields and the opt-in event. Existing draw-count state events and the default present,state subscription are unchanged. Sampling is off with HOS_CONTROL unset: one early return, no guest reads or allocation. No arbitrary-memory opt-in is needed for these named, read-only values.

How to try it

Start the game with HOS_CONTROL as described in docs/control.md, then:

python -I scripts/hookshot_ctl.py --json status
python -I scripts/hookshot_ctl.py --json watch --events oot --every 6 \
    --flags PlayerStatus.Life,0x31ab5580 --changes

Equivalent subscription:

{"id":11,"cmd":"subscribe","events":["oot"],"every":6,"flags":["PlayerStatus.Life","0x31ab5580"],"changes":true}

Events have event:"oot", the subscription's sub, and a nested game snapshot. Use game.frame, not the envelope's delivery frame, to date it. Samples stop wherever Link is not updated, including the title and pauses; status retains the last snapshot. Without a fresh sample, do not assume Link is still in control. The default cadence stays 60 presents; a steering client explicitly requests 6 here.

How it was tested

Environment: 2026-11-11 UTC; Arch Linux, kernel 7.2.15-1-omarchy; Core i9-13900K; RTX 4090, NVIDIA 615.78.08; gcc 16.2.1 and clang 23.1.3. Source base a5b955f. File binary d847c0c; socket binary a57f35f. Later commits d06e495 and 045fc13 change only the test fixture and evidence documentation, not runtime code. Both binaries used the same matching lift and cached hot objects through the build gate.

Game-free

• test_oot_state_tap.py with gcc and clang, warnings as errors, through the no-core wrapper:

  PASS: OoT socket sampler, preserved hash/constants/lock wiring, both table caps and no file switches
  

  Coverage retains the old values, typed name/hash lookup and collision, MurmurHash3 reference and main constants, both table limits, escaped strings, nulls and hook-lock wiring. It adds independent subscriptions, 6/12/default-60 cadence, pending samples, replacement/unsubscribe/disconnect, moved/resized tables, 32 changes plus overflow, and the largest escaped watch payload. The resize case changes a key that stays in its slot, so rehashing alone cannot make a broken reset pass.

• 38 compiled behavioral mutations fail with each compiler: missing fields, cadence, independent baselines, null/range and finite-value guards, both caps, hashing/collisions, change accounting, lifecycle, retained samples, buffer size and argument validation. Two additional source negatives per compiler check missing hook wiring and the old checkout: 80 expected negative failures total, not 80 compiled cases.

• test_control.py: 29 PASS groups with gcc and 29 with clang, including the existing transport/security/input checks.

• test_hookshot_ctl.py: 14 tests, OK. The same tests against the old client fail in the new status/watch checks (2 failures, 1 error).

• Whole suite through hs-idle.sh, using the dependency Python:

  437 passed, 1 failed, 20 skipped in 235 s (linux, 32 at once)
  

  The failure is tests/test_bake_mimo.py:223: AssertionError: the unreadable .zs was not reported. That test and nx2/bake_mimo.py are byte-identical to a5b955f; this PR does not change either. The base was not rerun, so this is an unchanged-source failure, not a separately reproduced baseline result. The sampler, bridge and client tests pass in the suite. The no-core check reported no coredumps. Rerun on its own afterwards, test_bake_mimo.py passes: twice on this head (the g++ and the clang++ run of the four checks) and once on the base's tree, so it failed once in the 32-at-once suite run under load and not since.

In the game

Each game loaded a fresh copy of the same harness save. No directional input was sent. The completed pair used one ordinary 197.8 s GPU hold, headless, 1920x1080 output with 1280x720 captures and RX_VK_PRECOMPILE=0. The machine was shared; the wall times below are run lengths, not performance measurements.

Run	Build	Wall time	Load average before → after (1 minute)	Result
First attempt, file only	d847c0c	121.1 s	333.81 → 222.50	State captured and game exited cleanly. My analysis parser expected the version at column zero rather than after [hos host], so it stopped before starting the socket game. Kept, not counted as a comparison.
Completed pair, file	d847c0c	69.4 s	84.04 → 131.22	Reference standing snapshot; Link visible in the inspected capture.
Completed pair, socket	a57f35f	128.3 s	131.22 → 148.89	All nine compared fields equal; watched Bool 0→1, with the same hash/value in changed.

The comparison script asserted exact equality of map, position, facing, camera, life, maximum life, rupees, prompt and music, before the flag-changing debug action in the throwaway run. It then checked the watched name and its independently computed hash in the socket event stream:

PASS: 9 standing fields identical; watched Bool and changed hash proven; hold 197.8s

All three games exited 0. Each had 0 fatal, crash, device-lost, kernel-driver-fault and NVRM lines.

Visual limit: I inspected the captures. The file run shows Link at the saved position. The socket run's last capture shows the hearts and controls on black, not Link; its log still reports shader jobs queued and draws left out as not ready. Shader caches were not matched or warmed for an image comparison. The run proves the state values and flag reporting, not rendering parity or the cause of the black picture.

Not tested

• Windows/named-pipe execution here, AMD, Intel/Steam Deck, real controllers, and long sessions.
• In-game pause/resume and map-transition sampling; game-free update gaps and stale snapshots are covered.
• Rendering parity, frame-rate or sampling-overhead claims. These were shared-load, non-exclusive runs with unmatched shader-cache state.
• Later upstream runtime changes: the measured pair is deliberately pinned to a5b955f. main has moved on since (ef81aee when this was pushed); the branch merges with it without a conflict, and nothing was rerun on the merge.

Risks

• These are samples from Link's update, not an atomic snapshot of every game thread. The debug lock serializes Link's workers and the debug tools; range checks and nulls cover individual unreadable values. The bridge never follows guest pointers on its own thread.
• changes:true scans four tables at the requested cadence. The 32-entry output cap does not make that scan free; it stays opt-in. Intermediate values between samples are not a history of every write.
• A retained status.game through a pause/load is not proof of current control. Clients must use fresh sample frames and known-false event/loading values before steering.

Overlaps

The overlap scan found 43 open PRs touching these files; the actual conflicts were read:

• feat(vk): cap compiler pools at physical cores #2414: only simultaneous row insertion at the top of docs/tested-configurations.md. Keep both its historical compiler-worker row and these run rows; no runtime hunk conflicts.
• docs: rewrite the docs: player docs in docs/user/, developer docs in docs/dev/, the readme as a front page #2561: modify/delete conflicts for docs/control.md and docs/development.md. Its replacement control documentation is docs/dev/automation.md. Move the new protocol text, development reference and their source-reading test checks with that reorganization; do not drop their coverage. The host/client changes merge automatically.
• The same-code-file PRs, including Add an MCP server for scripted in-game testing #1025, feat(runtime): add an opt-in local scene testing API #1543 and feat(scripts): a pipeline tour that plays the game for the tour's pipeline state list #2048, merge cleanly in the scan. That is a merge result, not joint runtime testing.

The local pre-push checks pass (commit identity, trailers, no game file, a test changed, the title). The range-diff against d847c0c keeps its two existing commits and adds the bridge migration, strengthened fixture and run evidence.

Assisted-by: GPT-6 Astra
Co-Authored-By: GPT-6 Astra noreply@openai.com

[mooki000]

Rebased onto main b29967bf so the workflows run here: they had waited for an approval on the old head, and main's game-free tests are green again since #2380. The patch is unchanged (git range-diff: the commit is equal), and its test passes on the new base.

[Joey-Magnus]

Nice, self-contained addition. Some specifics I checked:

• The file is new (host/oot_state_tap.c). nx2/build.py compiles every host/*.c, so nothing else needs wiring.
• The hook in hos_oot_lift_player runs under the existing cheats_lock.
• While HOS_OOT_STATE_FILE is unset, the cost is one getenv on the first call and then an early return.
• Every guest read goes through hos_guest_range, and nothing in the game is written.
• The camera path (0x84ca6bc0 → manager +32/+40 → camera +120) matches oot_gameplay_view in host/hos_main.c.
• The switches are documented in docs/development.md.
• fopen on a UTF-8 path is fine on Windows because of the exe's UTF-8 manifest; tests/test_utf8_paths.py passes.

tests/test_oot_state_tap.py passes here (gcc, -Werror). The red Windows check is tests/test_native_astc_texture_count.py, which main fixed in #2397. The appimage failure was Launchpad timing out. Neither comes from this PR.

Minor points, none of them blocking:

• minor host/oot_state_tap.c state_entry: the probe loop runs up to size iterations, where size comes from guest memory. state_put_changes caps it at 1 << 22, but this function doesn't. A torn or garbage read while a save loads could make one line walk billions of slots on a guest worker. Using the same cap here (if (size > (1u << 22)) return 0;) would keep a bad read from stalling the game.
• question Fields that read the same thing: #926's control socket status (now on main) and #2048's !cheat 998 already report map, event, loading and age. You offered to move the fields into status. A maintainer should decide whether to keep two channels. I'm fine with this file as it is, because a file stream suits HOS_INPUT_LIVE scripts.
• Not tested on Windows or AMD, as the description says. The code has nothing platform-specific beyond InterlockedCompareExchange64, which host/posix provides.

[hckrman101]

Hi, can you check if this is integrated with the control bridge?

[Joey-Magnus]

The code is clean (opt-in, read-only, every guest read checked, and the test passes).

To answer the earlier question: it isn't integrated with the control bridge. host/control.c already has status (event, loading, age) and subscribe (map, state), and docs/control.md describes the bridge as the one connection for tools. Since you offered, moving these fields into status/subscribe seems better than a second channel.

One fix either way: state_entry (host/oot_state_tap.c:107) walks up to size slots read from guest memory with no upper limit. The sibling function caps it at 1u<<22, and this one should too.

[hckrman101]

I read the code and the discussion. I agree with Joey-Magnus's review: the tap is opt-in, read-only, every guest read is range-checked, and the test is self-contained and declared (RUN_ALL_ARGS). I have nothing to add on correctness.

Two things are still open:

• Control bridge (my earlier question). Joey-Magnus confirmed it is not integrated: host/control.c already has status and subscribe, and docs/control.md calls the bridge the one connection for tools. @mooki000 offered in the description to move the fields there, but hasn't answered since. Whether to keep a second channel (a file stream, which does suit HOS_INPUT_LIVE scripts) or fold the fields into status/subscribe is a call for @NotRexed. I'm not approving over that question.
• state_entry probe cap. The loop in host/oot_state_tap.c still runs up to a size read from guest memory with no upper bound. state_put_changes caps it at 1 << 22; this one should too (Joey-Magnus's point, still unfixed at this head). Small change, no new test needed beyond the existing one.

CI: the Windows source-check failure is test_native_astc_texture_count.py, which main fixed in #2397 (this branch is based just before it), and appimage is a Launchpad timeout. Neither touches this PR. Once the cap is in and the channel question is settled, this is fine to merge.

[mooki000]

@hckrman101 @Joey-Magnus thanks both.

• The cap is in (d847c0cf): state_entry returns no entry for a table that reports more than 1 << 22 slots, the same limit as state_put_changes. The test's fixture sets such a size on a table whose first slot matches the flag asked for and expects no entry; it fails without the line, with gcc and clang. Rebased onto a5b955fb, so the Windows job has test: test_native_astc_texture_count builds on Windows, where the CRT marks scanf deprecated #2397's fix.
• Control bridge: not integrated at this head, as Joey-Magnus found. I'll do what I offered: move the fields into status/subscribe and drop the file switches, so the bridge stays the one connection for tools (unless @NotRexed wants the file kept). I'll push that here once a run in the game shows the same values over the socket as the file gives.

[hckrman101]

Re-review at d847c0c.

Fixed: the state_entry probe cap (Joey-Magnus's point). It now returns no entry for a table that reports more than 1 << 22 slots, the same limit as state_put_changes. I read the new check and the added test: the fixture accepts exactly 1 << 22 and rejects (1 << 22) + 1 even when the first slot matches, so it covers the case. Nothing else changed in the code; the rebase onto current main also picks up #2397's Windows fix.

Still open: the one-channel question. mooki000 confirmed the tap is not integrated with the control bridge and offered to move the fields into status/subscribe and drop the file switches, unless @NotRexed wants the file stream kept (it does suit HOS_INPUT_LIVE scripts). That is @NotRexed's call, so I'm not approving over it. If the fields move, this will need another look, since the file, its docs and the test would change a lot.

CI: at the time of writing most jobs are still running on the new head (builder package, appimage and actionlint are green). I found no problems in the code itself.

[mooki000]

@hckrman101 @Joey-Magnus the one-channel question, answered in code:

I moved the fields into the bridge in a57f35f: status.game is the snapshot; subscribe with events:["oot"], every, flags and changes streams it. The four file switches and file writing are gone. Reads stay in Link's locked update; the socket thread receives copies. The lookup cap from d847c0c remains, and d06e495 strengthens the resize-baseline check.

On Linux/RTX 4090, the same copied save gave identical map, position, facing, camera, life/max life, rupees, prompt and music over file versus socket. A watched Bool changed 0→1 and the same hash appeared in changed. Both games exited cleanly; no driver faults. gcc/clang checks pass, with 38 behavioral regressions rejected by each.

Windows and AMD are not tested. Nor is rendering parity: the socket capture showed HUD on black with shaders still queued. The full suite was 437 passed, 1 failed, 20 skipped; the failure was test_bake_mimo.py's unreadable-.zs assertion, a test and a script this change does not touch, and rerun on its own it passes here and on the base's tree. Details and all three runs are in the updated description.

@NotRexed if you would rather keep the file stream for HOS_INPUT_LIVE scripts, say so: the first two commits are still the file version as reviewed (d847c0c), and the move is the three commits above them, so it can come out again or the file can stay beside the bridge.

[mooki000]

Rebased the actual pushed head 045fc13d onto current main 87991bc8 as 723758ff. The only conflict was additive test-evidence rows; both contributors' records remain verbatim, with no runtime change, dropped commit or altered author/message/trailer. All 27 selected checks passed with GCC and Clang on the first rebase; the sampler, bridge and client controls failed against main's missing feature. Its suite was 470 passed / 1 inherited background-input fixture warning / 20 skipped. The final rebase adds only upstream #2671 and changes no PR file blob; the three feature checks, final-main negatives, #2671's offline check and Windows-target syntax all ran again. The final PR gate passes. The suite was not repeated, and no new gameplay, native Windows or performance evidence is claimed. Sources: the #2292 evidence and final-handoff ledger above.

[Joey-Magnus]

Both open points from the earlier round are settled. The state_entry cap is in, and the fields now go through status.game and an oot subscription instead of a second file channel, as hckrman101 and I asked. Guest reads still happen only in Link's update under cheats_lock. Only copied JSON crosses state_lock. Watch names are restricted to [A-Za-z0-9_.] or 0x plus 8 hex digits before they go into JSON. Every path that drops a connection clears its watch. I merged it into current main: test_oot_state_tap, test_control, test_hookshot_ctl, test_utf8_paths and test_env_on pass.

Minor points:

• host/oot_state_tap.c:351-371: a subscription with changes:true scans up to four GameData tables while holding state_lock on the game's worker, so a status call waits for that scan. Fine for a debug channel.
• status.game.event, loading and age can now be null. Before, they were always booleans or a string. docs/control.md says so, but scripts that test == false will notice.
• #2694 calls G.game->oot_state(&event, &loading, &child), which this PR replaces. Whichever lands second needs to adapt. #2561 also moves docs/control.md.