Skip to content

vk: the pipeline notes file and the state list opened by the precompile, before play, not at the first new pipeline in play - #1417

Merged
NotRexed merged 1 commit into
NotRexed:mainfrom
DUDUYG:perf/pipeline-notes-open
Nov 10, 2026
Merged

NotRexed merged 1 commit into
NotRexed:mainfrom
DUDUYG:perf/pipeline-notes-open

Conversation

@DUDUYG

@DUDUYG DUDUYG commented Nov 10, 2026

Copy link
Copy Markdown
Contributor

What

The pipeline notes file (hookshot_pipelines.bin) and the pipeline state list are now opened by the precompile, before play, not at the first new pipeline in play.

  • Notes file: pipeline_note opened it at the first pipeline of a run that was not prepared, on the replaying thread. It read every key back with a seek and an 8-byte read per record. With 1150 notes of 27 KB each, that took 142 ms: the one long "pipelines" stall early in play (150-290 ms in earlier runs).
  • State list: state_record read and hashed the whole list in the same draw, which took 8-18 ms.

Change (gpu/vk/vk_draw.cpp)

  • The opens moved out: each first-open moved into a function of its own, notes_open_locked and state_open_locked, which run with their mutexes held.
  • Called by the precompile: precompile_run calls both right after pipe_notes_read, so both files are open before the game draws.
    • The notes' keys come from the notes it has just read.
    • A note pipe_notes_read left out as not sane (note_sane) is written again when its pipeline is met, and the next read takes that one.
  • Fallback: where the precompile did not run, notes_open_locked reads the records in order, one seek past the head, a record at a time.
  • Switches: RX_VK_NO_PROGRAM_CACHE and RX_VK_NO_STATE_LIST are honoured as before.

Measured

  • Setup: RTX 5070 12 GB, Ryzen 7 9800X3D, Windows 11, NVIDIA 621.01, headless runs of a Kokiri Forest save over the same scripted 36 s walk.
  • Before: a diagnostic build timed the parts of a new pipeline's path. Its log read: a new pipeline cost the draw 151.0 ms: lock 0.0, the state list 8.5, the notes opened 141.9, written 0.0.
  • After: in 3 runs with this change, no new pipeline cost a draw 5 ms or more.
  • With vk: render target images reused from a pool, not made and destroyed for each of OoT's shifting transient targets #1416 (the render target image pool): no stall over 30 ms in play in those runs. The worst was 15.7-19.9 ms, at 59.5-60.0 fps.

Not covered

  • NVIDIA on Windows only. The first-open cost depends on the disk and on how warm the file is in the OS cache.
  • The notes file is still opened lazily, as before, when precompile_run does not run.

Tests

  • tests/test_pipeline_notes_open.py (source checks, 4 cases):
    • pipeline_note and state_record open through the shared functions, and no seek is done per record;
    • the keys come from the notes read, or from the records in order;
    • the precompile opens both before program_cache_load_some, under their mutexes, honouring the switches.
  • tests/run_all.py on every test that reads vk_draw.cpp: 32 pass. The 2 that fail here fail the same way on main on Windows: test_linux_driver_fault_guard.py needs POSIX signals, and test_precompile_compute_guard.py needs an MSVC-target clang++. The latter only extracts the precompile's compute arm, which this does not change.

🤖 Generated with Claude Code

…le, before play, not at the first new pipeline in play

pipeline_note opened hookshot_pipelines.bin at the first new pipeline of a run and read its keys back with a seek and an
8-byte read each: 142 ms for 1150 notes on the replaying thread (RTX 5070, Windows), the one long "pipelines" stall early
in play. state_record read and hashed the whole state list there too, 8-18 ms. precompile_run now opens both with the notes
it has just read (a note pipe_notes_read left out as not sane is written again when its pipeline is met); where it did not
run, the notes are read in order, a record at a time. tests/test_pipeline_notes_open.py keeps it so.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@NotRexed

Copy link
Copy Markdown
Owner

This landed in the v2.0.12 branch (merged there for testing; it reaches main with that release), so closing. Thanks!

@NotRexed NotRexed closed this Nov 10, 2026
@NotRexed

Copy link
Copy Markdown
Owner

Reopened: closing this was a mistake on my side. It was tested in the v2.0.12 branch, and it will be merged into main here, so it shows as merged. Sorry for the noise.

@NotRexed NotRexed reopened this Nov 10, 2026
@NotRexed
NotRexed merged commit da2a004 into NotRexed:main Nov 10, 2026
rakefire-impala added a commit to rakefire-impala/Hookshot-w-Mesh-Shader-Fallback that referenced this pull request Nov 10, 2026
…ompile

fallback_compute_pipelines made a fallback program's compute pipelines in
its first draw, under one lock. AMD's compiler took up to 12 s on one: every
recording thread stopped behind it and the game froze in the first room
with foliage, then the foliage and the door drew white (RX 5500M, RDNA1,
no VK_EXT_mesh_shader).

- fallback_pipelines_ready: the draw builds them on a worker, as dispatch()
  builds a compute program's pipeline (compute_state 0/1/2, which only
  compute programs used; RX_VK_COMPUTE_SERIAL as there). The draw that
  starts the build waits up to 50 ms for it, which covers NVIDIA's
  compiles; a draw while it still compiles is left out, as a dispatch is.
- fallback_compute_pipelines is the build. Its lock now covers NotRexed#1317's
  failure set and the program's pipelines, not the compiles, so several
  programs compile at once on the workers and in the precompile. A failure
  is still remembered by key and variant before the return, and not tried
  again (tests/test_mesh_fallback_failed_once.py, unchanged, passes).
- precompile_run builds every fallback program's pipelines with the
  compute programs', before play. program_cache_load_some takes a new
  fallbacks flag, which only the precompile sets, and then loads every
  program translated for the fallback (kind 4), not only those the notes
  name, whatever the compute list (NotRexed#1417) says: right after an update is
  when they are cold. Only a GPU without mesh shaders has such programs. The first start
  after a translator change compiles them (minutes on AMD; the precompile's
  stall rule lets the game start and they finish behind it); later starts
  take them from the driver's cache.

tests/test_mesh_fallback.py checks the worker, the draw,
the lock's scope, the precompile and the program cache's load.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
rakefire-impala added a commit to rakefire-impala/Hookshot-w-Mesh-Shader-Fallback that referenced this pull request Nov 10, 2026
…ompile

fallback_compute_pipelines made a fallback program's compute pipelines in
its first draw, under one lock. AMD's compiler took up to 12 s on one: every
recording thread stopped behind it and the game froze in the first room
with foliage, then the foliage and the door drew white (RX 5500M, RDNA1,
no VK_EXT_mesh_shader).

- fallback_pipelines_ready: the draw builds them on a worker, as dispatch()
  builds a compute program's pipeline (compute_state 0/1/2, which only
  compute programs used; RX_VK_COMPUTE_SERIAL as there). The draw that
  starts the build waits up to 50 ms for it, which covers NVIDIA's
  compiles; a draw while it still compiles is left out, as a dispatch is.
- fallback_compute_pipelines is the build. Its lock now covers NotRexed#1317's
  failure set and the program's pipelines, not the compiles, so several
  programs compile at once on the workers and in the precompile. A failure
  is still remembered by key and variant before the return, and not tried
  again (tests/test_mesh_fallback_failed_once.py, unchanged, passes).
- precompile_run builds every fallback program's pipelines with the
  compute programs', before play. program_cache_load_some takes a new
  fallbacks flag, which only the precompile sets, and then loads every
  program translated for the fallback (kind 4), not only those the notes
  name, whatever the compute list (NotRexed#1417) says: right after an update is
  when they are cold. Only a GPU without mesh shaders has such programs. The first start
  after a translator change compiles them (minutes on AMD; the precompile's
  stall rule lets the game start and they finish behind it); later starts
  take them from the driver's cache.

tests/test_mesh_fallback.py checks the worker, the draw,
the lock's scope, the precompile and the program cache's load.
NotRexed pushed a commit that referenced this pull request Nov 10, 2026
…ompile (#1512)

fallback_compute_pipelines made a fallback program's compute pipelines in
its first draw, under one lock. AMD's compiler took up to 12 s on one: every
recording thread stopped behind it and the game froze in the first room
with foliage, then the foliage and the door drew white (RX 5500M, RDNA1,
no VK_EXT_mesh_shader).

- fallback_pipelines_ready: the draw builds them on a worker, as dispatch()
  builds a compute program's pipeline (compute_state 0/1/2, which only
  compute programs used; RX_VK_COMPUTE_SERIAL as there). The draw that
  starts the build waits up to 50 ms for it, which covers NVIDIA's
  compiles; a draw while it still compiles is left out, as a dispatch is.
- fallback_compute_pipelines is the build. Its lock now covers #1317's
  failure set and the program's pipelines, not the compiles, so several
  programs compile at once on the workers and in the precompile. A failure
  is still remembered by key and variant before the return, and not tried
  again (tests/test_mesh_fallback_failed_once.py, unchanged, passes).
- precompile_run builds every fallback program's pipelines with the
  compute programs', before play. program_cache_load_some takes a new
  fallbacks flag, which only the precompile sets, and then loads every
  program translated for the fallback (kind 4), not only those the notes
  name, whatever the compute list (#1417) says: right after an update is
  when they are cold. Only a GPU without mesh shaders has such programs. The first start
  after a translator change compiles them (minutes on AMD; the precompile's
  stall rule lets the game start and they finish behind it); later starts
  take them from the driver's cache.

tests/test_mesh_fallback.py checks the worker, the draw,
the lock's scope, the precompile and the program cache's load.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants