perf(host): the game's short sleeps wait on the timer where the process has six logical processors or fewer by klimovich008 · Pull Request #2263 · NotRexed/Hookshot

klimovich008 · 2026-11-11T12:29:21Z

On a PC with few logical processors the game's idle job workers take processor time the frame's threads need: the frame rate is lower and some frames take a quarter of a second or more. This makes those workers really sleep there. Nothing changes with 8 logical processors or more.

Why it happens

OoT's job workers (ModuleSystemWorker*, OptimizeChangeBanc), with nothing to do, sleep 20 us and look at their queues again: about 7,600 SleepThread calls a second in Hyrule Field.
Windows cannot sleep 20 us: its high-resolution timer wakes about 0.5 ms late. So sleep_ns in host/hos_thread.c answers a sleep under a millisecond by yielding until its time (SwitchToThread in a loop). That is exact, and it keeps a processor busy for every sleeping thread.
With 16 logical processors that costs CPU time only. With 4 or 6, which is no more than the game's six cores, the yielding loops run instead of the threads that make the frame.

What changes

host/hos_thread.c: hos_short_sleep_timer decides once how sleeps under a millisecond wait. With six logical processors or fewer in the process's affinity mask, the game's own sleeps (nn::os::SleepThread and the kernel's SleepThread, both through hos_sleep_logged) wait on the timer. With more, and when the count is not known (the call failed, or the PC has more than one processor group, where the mask is one group's), they yield as before.
The host's own short sleeps (hos_sleep_ns: the frame limiter in nn_vi.c, the audio renderer, timer events) keep yielding in both cases, so pacing is not made half a millisecond late.
HOS_SHORT_SLEEP_TIMER has three states. Not set: the rule above. Set: it follows env_on's rule from fix: switches that behave as players expect, and dead ones removed #2042 (on: the timer for every short sleep, which is what the switch did before; 0, false, off, no or empty: yielding for every one).
The log says once when the timer is chosen, with the processor count.
The choice by processors is made on Windows only, where it was measured. Linux and the web build are as they were (the switch works there by its value).
tests/test_short_sleep_timer.py: the real hos_short_sleep_timer, sleep_away, hos_sleep_ns, sleep_ns and hos_sleep_logged, cut from the source, over stubs that count yields and timer waits, one process a case: 26 cases of processor count and switch value, the processor-group case, and six broken variants that must fail (the processors not looked at, the switch read by presence, the switch read as a number instead of by env_on's rule, the game's sleeps passed on as the host's, the host's sleeps put on the timer, one group's mask taken for the process's).

Tested

Windows 11, Ryzen 7 5700X3D (8 cores, 16 threads), Radeon RX 9070 XT (Vulkan 1.4, driver 0x8001a0), 2560x720 window (32:9), frame rate cap 60.

Fewer processors are emulated: the game is started with an affinity mask (inherited from its first instruction), on this one CPU. No PC with a 4- or 6-thread CPU was tested; such a CPU also has slower cores and less cache, and the renderer still sized its pools for 16 threads here.

Each cell is 4 pairs of runs on one binary, control HOS_SHORT_SLEEP_TIMER=0 (all yield, as main does when the switch is not set) against the switch unset (this PR's rule), alternating. A run measures 20 s from 3 s after the save has loaded: Hyrule Field with the camera turning (a player's save), Kokiri Forest standing (the default save). The log line confirms the rule in all 16 variant runs and in none of the control runs. The binary is main d8ce191 with this PR's change to hos_thread.c as it was before the merge with #2042 (so the PC's prepared shader caches matched; the merge changed only how a set switch's value is read). On the PR's head, merged with main b744b1e, I ran the tests and a whole-file syntax compile of host/*.c, not the game.

logical processors	scene	fps, yielding to the rule	CPU time a frame	p99 frame	worst frame of each run, yielding (ms)	with the rule (ms)
4 (mask 0x55: 4 cores)	Hyrule Field	47.0 to 51.3 (+9.2 %, 4 of 4 pairs)	64.7 to 55.6 ms (-14.0 %)	42.8 to 28.4 ms	279, 55, 729, 60	59, 62, 53, 53
	Kokiri Forest	58.0 to 58.9 (+1.7 %, 4 of 4)	50.1 to 45.6 ms (-8.5 %)	27.4 to 25.3 ms	422, 78, 53, 55	30, 30, 30, 30
6 (mask 0x555: 6 cores)	Hyrule Field	52.1 to 52.2 (+0.4 %, 4 of 4)	65.5 to 59.7 ms (-9.1 %)	26.3 to 25.0 ms	50, 55, 64, 49	51, 50, 54, 57
	Kokiri Forest	60.0 to 60.0	53.1 to 48.2 ms (-9.2 %)	22.1 to 21.8 ms	432, 26, 44, 31	28, 25, 25, 27

Frames over 250 ms: in 4 of the 16 yielding runs (2 of 4 in Hyrule Field on 4 processors), in none of the 16 runs with the rule.
Why the edge is at 6 and not higher, measured before this change with the old switch (HOS_SHORT_SLEEP_TIMER=1, every short sleep on the timer, release binary of main d8ce191, 4 pairs a cell):
- 4 processors, Hyrule Field: 50.4 to 52.8 fps (4 of 4); with yielding a frame over 250 ms in 5 of 16 runs, with the timer in none of 8.
- 6 processors: fps the same, CPU time a frame -12 %, frames over 100 ms in 2 of 4 yielding Hyrule runs and in none with the timer.
- 8 processors laid out as the host lays out a 4-core, 8-thread CPU (the Presentation Thread alone on the last core): no frame over 100 ms either way; Hyrule Field 45.5 to 46.1 fps, Kokiri Forest 59.5 to 59.3 fps (lower in 4 of 4 pairs). Left as it is.
- 16 processors: fps the same, CPU time a frame -5 to -13 %. Left as it is: nothing to gain in frame rate, and in Hyrule Field 3 of 32 runs with the timer (most of them with a shorter spin too) had a frame over 250 ms against 0 of 38 without (too few to tell).
A shorter spin before a wait (HOS_SPIN_US=5) was tried beside it on 4 processors: no effect. Not part of this PR.
Content, checked separately by a second agent (Codex): it compared this exact binary with the rule against =0 on 4 processors in Temple of Time, Hyrule Field and Kokiri Forest (frames captured every 120 to 600 frames, validation layers off): world and HUD intact in both arms, the only differences animation timing (the fairy, a title fading). Saves unchanged, no fatal line.
On the merged head: python -I -B tests/test_short_sleep_timer.py --cc gcc (MinGW gcc 16.2) and with clang, and tests/test_env_on.py: pass. The full tests/run_all.py on the merged head (Windows, MinGW gcc): 324 passed, 6 failed, 58 skipped. None of the six is this PR's: test_vcore_fs_open and test_vcore_video_acquire fail the same way on unchanged main b744b1e (fixed on main since by test: test_vcore_fs_open and test_vcore_video_acquire build again with env_on #2380), and test_fpx_sticky, test_pipeline_play_cache, nx2/test_nup_paths and test_precompile_compute_guard failed the same way on unchanged main ebbba2e on this PC.
Independent review (gpt-6.1-sol, read-only): one finding on the first version (processor groups), fixed with its own test case; none on the final head, and none on the merge with main.

Not covered

A real 4- or 6-thread CPU, Intel hybrid CPUs, and laptops: only an affinity mask on a 5700X3D.
Linux and the Steam Deck: the choice is not made there. The POSIX layer's timer is not Windows' and was not measured.
Longer play and other places than the two scenes; loading screens were not measured.
8 to 12 logical processors, where the timer for the game's sleeps alone was not measured (only the old all-sleeps switch).
NVIDIA and Intel GPUs (the change is on the CPU side; all runs were on one AMD card).

Kind of change: performance on small CPUs, a behaviour change in scheduling there (who runs when), default on only where measured, with the switch to turn it off (HOS_SHORT_SLEEP_TIMER=0). Related: #995 (low fps with the GPU not fully used); no report from that issue was reproduced.

🤖 Generated with Claude Code

…ss has six logical processors or fewer OoT's idle job workers sleep 20 us and look again, about 7,600 times a second. A sleep under a millisecond yields until its time, which keeps a processor busy for each sleeping thread. With no more processors than the game's six cores that is time the frame's threads need: held to 4 threads Hyrule Field had frames over 250 ms in 2 of 4 runs and none with the timer, at 47 against 51 fps; on 6 threads the frame rate stayed and a frame's CPU time fell 9 %. On 8 threads and more nothing changes. Only the game's own sleeps (SleepThread) take the timer there: the host's short sleeps (the frame limiter, the audio renderer) still come back at their time. HOS_SHORT_SLEEP_TIMER is read by its value: 1 is the timer for every short sleep, 0 is yielding for every one (it was read by its presence, so =0 turned the timer on). The choice is made on Windows only, where it was measured. Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>

main's NotRexed#2042 reads every on/off switch through env_on. HOS_SHORT_SLEEP_TIMER keeps its third state here (not set: the processors decide); when it is set, its value follows env_on's rule (on: the timer for every short sleep; 0, false, off, no or empty: yielding for every one), where this branch read it as a number. HOS_SLEEP_LOG is main's env_on, with the game's sleep kept behind it. Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>

klimovich008 · 2026-11-11T17:08:17Z

Merged main (b744b1e) to resolve the conflict with #2042 in host/hos_thread.c. HOS_SHORT_SLEEP_TIMER keeps its third state (not set: the processor count decides); when it is set, its value now follows env_on's rule, where this branch read it as a number (so =on would have been off). HOS_SLEEP_LOG is main's env_on. The test has a broken variant for that (the switch read as a number must fail), and tests/test_env_on.py passes. The description is updated.

🤖 Generated with Claude Code

hckrman101

Looks good to me. The split is the right one: only the game's own SleepThread waits go on the timer, the host's frame limiter and audio keep the exact yield, and the rule only kicks in at 6 logical processors or fewer, with HOS_SHORT_SLEEP_TIMER as an escape hatch both ways. The numbers back it up: the 250+ ms frames go away on 4 threads, and fps and CPU time a frame either improve or stay the same on 6.

What I checked:

Timer fallback: if CreateWaitableTimerExW(..., CREATE_WAITABLE_TIMER_HIGH_RESOLUTION, ...) fails (Windows older than 10 1803), a short sleep falls through to the same SwitchToThread loop, so nothing breaks and you just keep today's behaviour (see the inline nit about the log). timeBeginPeriod(1) in hos_main.c doesn't matter for this path, since the high-resolution timer doesn't depend on it.
short_timer is a plain static that several threads may start concurrently, but they all compute the same value, and the log line is guarded by InterlockedExchange. That's fine.
Both guest entries (nn::os::SleepThread and svc 0x0B) go through hos_sleep_logged → sleep_away(ns, 1). hos_sleep_ns stays host-side.
The processor-group check (GetActiveProcessorGroupCount() == 1) is right: on a multi-group machine the count is left unknown and sleeps yield as before.
CI: test_short_sleep_timer.py and test_env_on.py pass on Windows and Ubuntu. The red checks aren't caused by this PR: test_native_astc_texture_count fails the same way on main's latest source run (7109693), and the appimage job timed out installing packages.

Linux parity and #2313: the two PRs rewrite the same short_timer lines in sleep_ns, so whichever lands second needs a rebase. #2313 is also based on main from before #2042 (it reads the switch by presence and atoi, not env_on). They don't overlap in intent, though. This PR leaves Linux yielding unless the switch is set, while #2313 makes the timer the default on Linux, where clock_nanosleep wakes within ~50 us (measured on the Deck: kernel time per worker 0.4-0.6 s/s → 0.04). I'd land this one first, then rebase #2313 onto hos_short_sleep_timer: on non-Windows, an unset switch would return 2 (all sleeps on the timer) instead of 0, and the new test would need one case for it. @Kolkka FYI.

Nits only. Approving.

… asked for, not made (review of NotRexed#2263) The line is logged when the choice is made, before any thread has created its timer. Where CreateWaitableTimerExW(HIGH_RESOLUTION) fails (Windows before 10 1803) a short sleep still goes through the yield loop at the end of sleep_ns, so "wait on the timer" overstated it there. Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>

hckrman101

Re-approving at d106855. The only change since my approval is the nit fix: the log line now says "where Windows has the high-resolution one" and the comment above it explains the pre-1803 fallback to the yield loop. That covers it, nothing new.

CI is green except Windows part 1, where only tests/test_native_astc_texture_count.py fails. That test is unrelated to this change (it doesn't touch host/hos_thread.c), so I'd treat it as main's flake or an existing failure. Worth a re-run before merging.

…stead of yielding (#2637) **Draft.** On Linux, OoT's idle job workers burn a processor each spinning in a yield loop while one of them waits on the GPU. This makes the game's own sub-millisecond sleeps (`SleepThread`) wait on the timer instead, as Windows already does on 6-thread CPUs since #2263. ## What it costs today With #2602, Hyrule Field (RTX 3090, Ryzen 9 7900X, 24 threads, Linux, default options): - The five `ModuleSystemWorker` threads and `sead::MainThread` yield for 3.3-3.6 ms a frame each in `sleep_away`. That is about 9 ms of CPU a frame in all, while one of them waits on a GPU readback. - The cause: on Linux `sleep_ns` never counts the processors, so every sleep under a millisecond is a yield loop (`host/hos_thread.c`). ## The change On Linux, when `HOS_SHORT_SLEEP_TIMER` is not set, the game's own short sleeps wait on the timer (mode 1). The host's own short sleeps (the frame limiter, the audio renderer) keep yielding, so frame pacing is unchanged. - `HOS_SHORT_SLEEP_TIMER=0`: all yield, as before. - `HOS_SHORT_SLEEP_TIMER=1`: all on the timer. - Windows is unchanged. ## Measured Hyrule Field, one run each, all sleeps on the timer (`HOS_SHORT_SLEEP_TIMER=1`, mode 2, which also moves the host's sleeps). This PR's default (mode 1) has not been run yet. | | median frame | fps | p99 | CPU a frame, all threads | |---|---|---|---|---| | yield (main) | 11.30 ms | 84.2 | 17.4 ms | 44.8 ms | | timer for all | 11.32 ms | 81.8 | 16.3 ms | **35.4 ms** | - **The same median frame, and about 9 ms less CPU a frame** (−21 %). An earlier run on main before #2602 also gave 37.5 ms against 64-73 ms. - On this 24-thread CPU the saving is CPU only. On 4-8 thread CPUs (the Steam Deck has 8) those yield loops take processors the frame needs; #2263 measured that on Windows (Hyrule Field 47 → 51 fps on 4 threads, and its long frames gone). - **The cost:** a job worker wakes about 0.5 ms later. In Hyrule Field the job workers are not on the frame's critical path (`HOS_CHAIN`). #2263 found TOTK's 20 µs waits on its critical path with the timer for all sleeps, which is why this changes only the game's own sleeps. ## Tested - `tests/test_short_sleep_timer.py` passes. It covers `hos_short_sleep_timer` itself, which is unchanged; the new Linux default is in its caller. - A game build of this commit is running. ## Not yet tested - **This PR's mode** (the game's sleeps only) in the game, as an A/B against main: fps, frames over 20/33 ms, CPU a frame. Hyrule Field and Kokiri, uncapped and at a 60 cap. - 4-8 thread CPUs and the Steam Deck, where it should matter most. Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>

…ns on 7+ thread CPUs (#2709) On CPUs with more than six logical processors, OoT's idle job workers each keep a processor busy. They poll in 20 µs steps through a yielding short sleep, and spin 20 µs before every event, queue or mutex wait sleeps. This PR limits how many of them poll at once, and shortens their spin, so the frame uses less CPU for the same picture. ## The change Both are default on, with a switch to restore the old behaviour. Windows only for the first; the second is on every platform. - **Short sleeps (`host/hos_thread.c` `sleep_ns`).** Where the game's sleeps under 1 ms yield (Windows, 7+ logical processors; ≤6 already use the timer since #2263), at most `HOS_SHORT_SLEEP_POLLERS` game threads at a time yield through one. The default is a quarter of the logical processors (4 on 16). The next one waits on the high-resolution timer until a slot is free. - The frame's serial threads (the game thread, the Presentation Thread) and the host's own sleeps (frame limiter, audio) always yield, as before. - `HOS_SHORT_SLEEP_POLLERS=0`: all yield, as before. `HOS_SHORT_SLEEP_TIMER` set: no limit (it already decides for all). - Linux is unchanged: since #2637 the game's sleeps there wait on the timer anyway. - **Spin before a wait sleeps (`hos_spin_ns`, used by `nn_os_sync.c` and `hos_svc.c`).** The serial threads keep `HOS_SPIN_US` (20 µs). The others spin `HOS_SPIN_OTHER_US` (5 µs, never more than `HOS_SPIN_US`). - A sleeper runs again 6.6–8.4 µs after its signal, so 5 µs still catches the quick hand-offs. - `HOS_SPIN_OTHER_US=20`: one spin for all, as before. - A thread is "serial" by the name the game gives it (`hos_thread_set_name`); the flag is atomic. The choices are read once and published as one atomic word each. ## Measured **Provisional:** measured on 57ed3aa, which predates #2602's fix of a main regression (41 → 53 fps on this test); to be re-measured on 4678050. Windows 11, Ryzen 7 5700X3D (16 logical processors), RX 9070 XT, AMD driver 26.10.2. Standard Hyrule Field test: right stick held right from the load, `HOS_FPS_MAX=240`, 2560x720 (32:9), 20 s window, warm caches. Medians of 6 alternating pairs. Base is main at 57ed3aa; the candidate is the same tree with this PR and #2708 merged. | arm | fps | CPU ms a frame | p99 | worst | runs with a frame over 100 / 250 ms | vcore escapes | |---|---|---|---|---|---|---| | main | 39.6 | 128.4 | 33.2 | 65.5 | 0 / 0 | 6 | | both of this PR + the Signal early return | **40.1** | **122.8 (−4.4 %)** | 32.6 | 81.4 | 0 / 0 | 0 | | short-sleep limit only | 39.85 | 124.9 (−2.7 %) | 32.8 | 578.8 | 1 / 1 | 6 | | 5 µs spin only | 40.15 | 124.5 (−3.1 %) | 31.9 | 493.7 | 2 / 2 | 0 | - CPU a frame was lower in all 6 pairs for the full candidate; fps changed little (+1.3 %): at 16 threads the saving is CPU, not frame time. - The long frames in the single-change arms all fell in pairs 0–1, the first runs after a reboot. A warm-up run on main had a 382 ms frame there too. From pair 2 on, no arm had a frame over 100 ms. - The game clock read 1.000 between the same two short dips (loads/turns) in every arm. ### 8 and 6 threads The same test, with the game held to 8 (`0xFF`) or 6 (`0x3F`) logical processors from its start. Each cell has 6 alternating pairs. Base is main 57ed3aa; the candidate is this PR + #2708, built as packs over it. | threads | arm | fps | CPU ms a frame | p99 | worst | runs with a frame over 100 / 250 ms | vcore escapes (whole runs) | |---|---|---|---|---|---|---|---| | 8 | main (5 runs; one rejected: no complete window) | 33.5 | 154.1 | 42.5 | 654.4 | 1 / 1 | 8 | | 8 | this PR + #2708 | **33.85** | **115.5 (−25 %)** | 40.8 | 65.1 | 0 / 0 | 8 | | 6 | main | 33.8 | 79.6 | 39.2 | 102.0 | 1 / 0 | 7 | | 6 | this PR + #2708 | 34.0 | 77.2 (−3.1 %) | 39.9 | 607.8 | 1 / 1 | 12 | - **8 threads:** CPU a frame was lower in all 5 complete pairs. The short-sleep limit is 2 there, so most idle workers wait on the timer instead of yielding. fps changed little: the frame is not CPU-bound at 8 threads either. - **6 threads:** the short-sleep limit does not act (≤6 already uses the timer since #2263), so only the 5 µs spin and #2708 differ. CPU was lower in 5 of 6 pairs. - **The 608 ms frame at 6 threads:** one candidate run, pair 4, has it. Main's run in pair 5 was also degraded (102 ms worst, 16 frames over 50 ms), and the gap between those pairs shows other jobs ran on the machine in between. One cell cannot separate it from that load. A second 6-thread cell is queued to check it. ## Tested - `tests/test_short_sleep_timer.py` (extended): the limit's choice over `HOS_SHORT_SLEEP_POLLERS` values and processor counts, run on the real `sleep_ns` block. Broken variants (no limit, serial threads counted, a leaked count) must fail. - `tests/test_spin_by_role.py` (new): `hos_spin_ns` from the source over the switch values, including negatives and huge values. Broken variants (no role, no clamp, no off, the Presentation Thread not serial) must fail. Wiring checks: every spin asks `hos_spin_ns`. - `tests/run_all.py`; syntax compile of every changed `host/*.c`; a gpt-6.1-sol review (three rounds, last clean). ## Not tested - 4-thread CPUs; real 6- and 8-thread CPUs (measured here by holding a 16-thread CPU to 6 or 8). Linux and the Steam Deck. - Scenes other than Hyrule Field; a 60 fps cap. 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>

klimovich008 and others added 2 commits November 11, 2026 12:56

hckrman101 approved these changes Nov 11, 2026

View reviewed changes

Comment thread host/hos_thread.c Resolved

hckrman101 approved these changes Nov 11, 2026

View reviewed changes

hckrman101 mentioned this pull request Nov 11, 2026

feat(vk): cap compiler pools at physical cores #2414

Draft

hckrman101 merged commit 4d488bc into NotRexed:main Nov 11, 2026
11 of 12 checks passed

github-actions Bot mentioned this pull request Nov 11, 2026

chore(main): release 2.2.0-rc.1 #2071

Closed

This was referenced Nov 11, 2026

perf(vk): reduce renderer CPU work and add bounded diagnostics #1541

Open

fix(linux): sleep short waits on the timer instead of yielding #2313

Closed

hckrman101 mentioned this pull request Nov 12, 2026

perf(host): on Linux the game's own short sleeps wait on the timer instead of yielding #2637

Merged

klimovich008 mentioned this pull request Nov 12, 2026

perf(host): fewer idle game threads poll through short sleeps and spins on 7+ thread CPUs #2709

Merged

Navigation Menu

Repository navigation

perf(host): the game's short sleeps wait on the timer where the process has six logical processors or fewer - #2263

perf(host): the game's short sleeps wait on the timer where the process has six logical processors or fewer#2263
hckrman101 merged 3 commits into
NotRexed:mainfrom
klimovich008:perf/short-sleep-timer-few-threads

klimovich008 commented Nov 11, 2026 •

edited

Loading

Uh oh!

klimovich008 commented Nov 11, 2026

Uh oh!

hckrman101 left a comment

Uh oh!

Uh oh!

hckrman101 left a comment

Uh oh!

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

klimovich008 commented Nov 11, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Why it happens

What changes

Tested

Not covered

Uh oh!

klimovich008 commented Nov 11, 2026

Uh oh!

hckrman101 left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

hckrman101 left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

klimovich008 commented Nov 11, 2026 •

edited

Loading