Skip to content

perf(vk): back-to-back passes and dispatches ordered by the game's barriers (NVIDIA by default) - #2559

Draft
NotRexed wants to merge 1 commit into
mainfrom
perf/game-barrier-order
Draft

NotRexed wants to merge 1 commit into
mainfrom
perf/game-barrier-order

Conversation

@NotRexed

Copy link
Copy Markdown
Owner

Why. Every draw pass begin and every compute dispatch records a full barrier, which makes the GPU drain everything before each of them. The game already records its own barrier (nvnCommandBufferBarrier, which the replay keeps as a full barrier) wherever a pass samples what an earlier one drew or a dispatch reads what another wrote. The extra barriers between back-to-back work are pure GPU idle time: elsewhere it measured 2.3 ms of a 9.7 ms GPU frame in Kokiri Forest, and about a tenth off the GPU's frame time on an RTX 5090.

What (RX_VK_GAME_ORDER):

  • A draw pass right after another, with nothing recorded between, waits in its fragment stages only. This is the existing RX_VK_PASS_BARRIER=game path, now on with the switch. RX_VK_PASS_BARRIER=game / full still decide that part on their own.
  • A direct dispatch right after another gets no barrier. "Right after" means the same command buffer with no GPU work recorded between, checked through the work_epoch count taken right after the previous dispatch's command.
  • Still full barriers: a switch between draws and compute, the replay's own copies, clears, fills and uniform updates (all count as work), indirect dispatches (their grid may come from the dispatch before), a parallel replay, and the game's barriers.
  • Defaults: on for NVIDIA (vendor 0x10de), since the game's barriers were placed for an NVIDIA GPU. Off on other vendors, where it hasn't run: RX_VK_GAME_ORDER=1 to try it there, =0 to turn it off everywhere.
  • Logging: the log states the choice once, and counts the dispatches left without a barrier every 50,000.

Checks

  • tests/test_game_order.py: the rules as written.
  • The barrier-related tests pass (mesh_fallback, dump_readback, present_aspect, upscale_fallback, vulkan_clear_mip).
  • vk_texture.cpp compiles. vk_draw.cpp needs Boost, which this machine lacks, so CI compiles it.

Not yet verified in-game: GPU frame time A/B on this NVIDIA card plus the flash check (every-frame shots).

🤖 Generated with Claude Code

…rriers (NVIDIA by default)

Every pass begin and every compute dispatch recorded a full barrier, which makes the GPU finish everything before each
of them. The game records its own barrier (nvnCommandBufferBarrier, kept as a full barrier) wherever a pass samples what
an earlier one drew or a dispatch reads what another wrote. With RX_VK_GAME_ORDER: a draw pass right after another
waits in its fragment stages only (the existing RX_VK_PASS_BARRIER=game path), and a direct dispatch right after
another, nothing recorded between, gets no barrier. A switch between draws and compute, the replay's own copies,
clears, fills and uniform updates, indirect dispatches, a parallel replay and the game's barriers keep the full one.
On NVIDIA by default (the game's barriers were placed for an NVIDIA GPU), off on others: =1 to try it there, =0 off.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@PedroFlaminio

Copy link
Copy Markdown

Tested in the game on NVIDIA (on by default there): no change in frame rate here; this machine is CPU-bound in this scene.

Hardware: RTX 3060 12 GB, NVIDIA 615.71.09, Manjaro (kernel 6.12), Core Ultra 7 265KF (8 P-cores + 12 E-cores), Wayland. Built on main 51da0464: each PR merged onto it, only hookshot_vk.so built (container build, RX_MARCH=x86-64-v3 RX_RELEASE=1) and swapped into the same game folder; main's hookshot_vk.so built the same way as the baseline.
Settings: 3440x1440 monitor, picture 2576x1080 (Aspect ratio: monitor's), frame interpolation 2x, Reflex on, V-Sync off, no frame rate cap; otherwise unchanged between runs.
Scene: Hyrule Field from the same save, standing, no input, 3.5 to 7 minutes a run; only the 5 s windows after loading (3,370-3,760 draws in every run) are counted. One run each, in the order main, #2633, #2644, #2559.

renderer fps median (min-max) replay submit draw descriptors memory sync
main 42.7 (41.4-43.4) 21.4 ms 9.4 3.5 9.4
#2633 42.8 (40.6-43.8) 19.9 ms 7.6 2.7 9.4
#2644 42.6 (40.2-43.5) 21.2 ms 8.9 2.9 9.5
#2559 43.0 (42.3-43.8) 20.1 ms 7.7 2.7 9.6

(replay columns: medians of the last 20 a frame's replay time lines; GPU-bound wait was 0.0 ms in all runs)

  • The log confirms it was on: passes and dispatches right after others ordered by the game's barriers: on, and dispatches left without a barrier were counted (over 50,000).
  • GPU-bound wait was 0.0 ms in every run, so a shorter GPU frame would not show in the frame rate here; I did not measure GPU time.
  • No device lost, and no visual problem seen while playing; I did not check for flashes frame by frame (no every-frame screenshots).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants