Skip to content

perf(host): fewer idle game threads poll through short sleeps and spins on 7+ thread CPUs - #2709

Open
klimovich008 wants to merge 5 commits into
NotRexed:mainfrom
klimovich008:perf/os-sync-polls
Open

klimovich008 wants to merge 5 commits into
NotRexed:mainfrom
klimovich008:perf/os-sync-polls

Conversation

@klimovich008

@klimovich008 klimovich008 commented Nov 12, 2026 •

Copy link
Copy Markdown
Contributor

On CPUs with more than six logical processors, OoT's idle job workers each keep a processor busy. They poll in 20 µs steps through a yielding short sleep, and spin 20 µs before every event, queue or mutex wait sleeps. This PR limits how many of them poll at once, and shortens their spin, so the frame uses less CPU for the same picture.

The change

Both are default on, with a switch to restore the old behaviour. Windows only for the first; the second is on every platform.

  • Short sleeps (host/hos_thread.c sleep_ns). Where the game's sleeps under 1 ms yield (Windows, 7+ logical processors; ≤6 already use the timer since perf(host): the game's short sleeps wait on the timer where the process has six logical processors or fewer #2263), at most HOS_SHORT_SLEEP_POLLERS game threads at a time yield through one. The default is a quarter of the logical processors (4 on 16). The next one waits on the high-resolution timer until a slot is free.
  • Spin before a wait sleeps (hos_spin_ns, used by nn_os_sync.c and hos_svc.c). The serial threads keep HOS_SPIN_US (20 µs). The others spin HOS_SPIN_OTHER_US (5 µs, never more than HOS_SPIN_US).
    • A sleeper runs again 6.6–8.4 µs after its signal, so 5 µs still catches the quick hand-offs.
    • HOS_SPIN_OTHER_US=20: one spin for all, as before.
  • A thread is "serial" by the name the game gives it (hos_thread_set_name); the flag is atomic. The choices are read once and published as one atomic word each.

Measured

Provisional: measured on 57ed3aa, which predates #2602's fix of a main regression (41 → 53 fps on this test); to be re-measured on 4678050.

Windows 11, Ryzen 7 5700X3D (16 logical processors), RX 9070 XT, AMD driver 26.10.2. Standard Hyrule Field test: right stick held right from the load, HOS_FPS_MAX=240, 2560x720 (32:9), 20 s window, warm caches. Medians of 6 alternating pairs. Base is main at 57ed3aa; the candidate is the same tree with this PR and #2708 merged.

arm fps CPU ms a frame p99 worst runs with a frame over 100 / 250 ms vcore escapes
main 39.6 128.4 33.2 65.5 0 / 0 6
both of this PR + the Signal early return 40.1 122.8 (−4.4 %) 32.6 81.4 0 / 0 0
short-sleep limit only 39.85 124.9 (−2.7 %) 32.8 578.8 1 / 1 6
5 µs spin only 40.15 124.5 (−3.1 %) 31.9 493.7 2 / 2 0
  • CPU a frame was lower in all 6 pairs for the full candidate; fps changed little (+1.3 %): at 16 threads the saving is CPU, not frame time.
  • The long frames in the single-change arms all fell in pairs 0–1, the first runs after a reboot. A warm-up run on main had a 382 ms frame there too. From pair 2 on, no arm had a frame over 100 ms.
  • The game clock read 1.000 between the same two short dips (loads/turns) in every arm.

8 and 6 threads

The same test, with the game held to 8 (0xFF) or 6 (0x3F) logical processors from its start. Each cell has 6 alternating pairs. Base is main 57ed3aa; the candidate is this PR + #2708, built as packs over it.

threads arm fps CPU ms a frame p99 worst runs with a frame over 100 / 250 ms vcore escapes (whole runs)
8 main (5 runs; one rejected: no complete window) 33.5 154.1 42.5 654.4 1 / 1 8
8 this PR + #2708 33.85 115.5 (−25 %) 40.8 65.1 0 / 0 8
6 main 33.8 79.6 39.2 102.0 1 / 0 7
6 this PR + #2708 34.0 77.2 (−3.1 %) 39.9 607.8 1 / 1 12

Tested

  • tests/test_short_sleep_timer.py (extended): the limit's choice over HOS_SHORT_SLEEP_POLLERS values and processor counts, run on the real sleep_ns block. Broken variants (no limit, serial threads counted, a leaked count) must fail.
  • tests/test_spin_by_role.py (new): hos_spin_ns from the source over the switch values, including negatives and huge values. Broken variants (no role, no clamp, no off, the Presentation Thread not serial) must fail. Wiring checks: every spin asks hos_spin_ns.
  • tests/run_all.py; syntax compile of every changed host/*.c; a gpt-6.1-sol review (three rounds, last clean).

Not tested

  • 4-thread CPUs; real 6- and 8-thread CPUs (measured here by holding a 16-thread CPU to 6 or 8). Linux and the Steam Deck.
  • Scenes other than Hyrule Field; a 60 fps cap.

🤖 Generated with Claude Code

klimovich008 and others added 4 commits November 12, 2026 08:20
…s yield through a short sleep at once

Where the game's short sleeps yield (7+ logical processors), OoT's idle job workers each keep a processor busy polling in
20 us steps. At most HOS_SHORT_SLEEP_POLLERS of the game's threads (default: a quarter of the logical processors) yield
through one at a time; the next waits on the high-resolution timer until one is back. The frame's serial threads (the
game thread, the Presentation Thread) and the host's own sleeps always yield. HOS_SHORT_SLEEP_POLLERS=0: as before.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ial threads

The 20 us spin before an event, queue or mutex wait sleeps cost 15 % of a core in Hyrule, 90 % of it on the job
workers and actor executors; a sleeper runs again 6.6-8.4 us after its signal. The game thread and the Presentation
Thread keep HOS_SPIN_US (20 us); the others spin HOS_SPIN_OTHER_US (5 us, at most HOS_SPIN_US). One reader for both
spin sites (nn_os_sync.c, hos_svc.c). HOS_SPIN_OTHER_US=20: as before.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…mic word each; the serial flag atomic

(sol review: a thread could see the choice made and the cap not yet; racing first callers wrote the spin lengths
unsynchronized; SetThreadName can name another thread while it reads its flag. The no-limit broken variant now keeps
the count.)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
(sol review round 2: a huge HOS_SPIN_OTHER_US overflowed the multiplication.)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@klimovich008

Copy link
Copy Markdown
Contributor Author

The 8-thread (limit 2) and 6-thread cells of the same Hyrule Field test are queued; I'll add them to the description with fps, CPU a frame and long-frame counts. If 8 threads shows more long frames, the default goes off below 13 threads.

@Brett-D

Brett-D commented Nov 12, 2026 •

Copy link
Copy Markdown
Contributor

Tested 75ce5c0c on a 24-thread i7-13700K (8P+8E), RTX 5090 / NVIDIA 621.14, Windows 11, 96 GB RAM. This isolates #2709 on main c544828f, without #2708, with the same renderer, image, warm caches and copied saves/options.

Kokiri Forest autosave, stationary, 2048x1152, frame generation off, HOS_FPS_MAX=240, windowed/muted/scripted input. Warm-up per arm, then three rounds in ABC/CBA/ABC order, followed by three more cap2/main pairs in BA/AB/BA order (six pairs total, balanced order). Frame statistics: 40-70 s after first present; process CPU: wall 45-75 s after launch.

Defaults, three matched runs each:

Arm FPS p50 / p95 / p99, ms Process cores busy
Main 67.73 14.52 / 17.47 / 19.11 6.01
#2709 defaults (logged cap 6) 68.20 14.51 / 16.67 / 18.31 5.92

Cap 2, six matched runs each:

Arm FPS p50 / p95 / p99, ms Process cores busy
Main 67.70 14.55 / 17.22 / 19.19 6.00
#2709, HOS_SHORT_SLEEP_POLLERS=2 66.52 14.85 / 17.38 / 19.14 5.38

Defaults saved only 0.09 cores here, within run-to-run noise; the five ModuleSystemWorker threads' CPU barely changed. With cap 2, the saving was 0.623 cores (10.4%), beyond the shared comparison's 2-SD noise gate, and each worker fell 10.9-12.1 percentage points of one core. The FPS and tail changes remain below that gate, but cap 2's mean FPS was 1.7% lower, with four of six paired deltas negative. This is not proof of no FPS regression, and I would not use this result to tighten the default. No errors, >50 ms frames, stalls, or concurrent-game/compiler interference in either PR batch. Linux gameplay/Steam Deck not tested here.

One Clang 23 fixture issue: the no-clamp mutation generates other = other;, which fails compilation under -Werror,-Wself-assign instead of exercising the broken runtime case. Removing the whole clamp statement fixes that without suppressing a warning or weakening the check:

broken("no-clamp", "@SPIN@", "other = other > s ? s : other;", "", (None, "50", 20000, 20000),
       "the others spinning longer than HOS_SPIN_US passes")

With that fixture-only repair, the Windows full suite gives 456 passed / 35 skipped / 1 unrelated baseline failure (test_input_background.py, Clang 23 unused oc_consumed_pads). Both of this PR's fixtures pass.

@klikso

klikso commented Nov 12, 2026

Copy link
Copy Markdown
Contributor

A 12-thread cell (limit 3), on NVIDIA: the same frame rate for about 9% less CPU.

Windows 11, Ryzen 5 5600X (6 cores, 12 logical processors), RTX 3060 Ti 8 GB (driver 32.0.16.2114), 2560x1440 DLSS. main ba005ca with #2711, #2731 and #2685 merged locally (both arms), plus this PR (without #2708). A/B with HOS_SHORT_SLEEP_POLLERS=0 HOS_SPIN_OTHER_US=20 as the old behaviour. Hyrule Field child, 40 s of the camera turning after a settle, RX_VK_FRAMEGEN=0, HOS_FPS_MAX=240, warm caches, arms alternated. CPU is the game process's user + kernel time over the measured window (GetProcessTimes), divided by the frames shown.

fps median fps mean CPU ms a frame frame p99 frames over 50 ms
this PR 60.8 / 59.4 / 60.1 60.3 / 58.5 / 59.0 47.0 / 47.6 / 49.8 24.9 / 31.7 / 28.7 ms 0 / 0 / 0
old (=0, =20) 60.1 / 61.0 / 55.5 58.9 / 60.4 / 55.4 51.1 / 50.6 / 56.6 31.3 / 27.1 / 33.2 ms 0 / 0 / 1

No more long frames with it than without on 12 threads. The third old run is the slowest of the six; without it the fps arms are level and CPU a frame is still ~6% lower.

@Joey-Magnus Joey-Magnus left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I merged this with #2708 into current main and ran tests/run_all.py. Only test_lift_cache.py fails, and it fails on main here too. test_short_sleep_timer.py and test_spin_by_role.py pass with clang 21. The code reads correctly. The poller count is always decremented, the choice and the spin lengths are each published as one word, and Linux gets no cap because threads stays 0 there.

This turns on by default for every Windows player with more than 6 logical processors, so I think a maintainer should decide on the evidence:

  • Your 16- and 8-thread cells and klikso's 12-thread cell show 4-25 % less CPU per frame at the same fps.
  • Brett-D's 24-thread run shows the default cap of 6 within noise. Cap 2 there cost 1.7 % fps.
  • The 608 ms frame in your 6-thread cell is still unexplained. Only the 5 us spin differs there.

Still open from Brett-D: the no-clamp mutation at tests/test_spin_by_role.py:130 produces other = other;, which fails to compile under clang 23 with -Werror,-Wself-assign. His replacement fixes it. Also, in host/hos.h:113 the "one of the frame's serial threads" comment trails hos_spin_ns instead of hos_thread_serial.

…); hos.h's serial-thread comment on hos_thread_serial

The no-clamp broken variant turned the clamp into `other = other;`, which clang 23 rejects under -Werror,-Wself-assign
instead of running the broken case. It now removes the whole statement (Brett-D's replacement). The "one of the frame's
serial threads" comment in host/hos.h trailed hos_spin_ns; it belongs to hos_thread_serial.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants