[SadTransGirl]

perf(vk): sparse VRAM mirrors, memory only where a game pool is (RX_VK_MIRROR_SPARSE=1)

On cards short of video memory the buffer pools' VRAM mirrors hold hundreds of MB that no game pool is in. This makes them sparse, opt-in. A 6 GB card's field walk went from 47.7-51.5 fps to 55.3-58.2 fps over three pairs. At 8 GB and more there's no fps change, but the VRAM peak is lower. Loads are no slower.

Cause. A pool's mirror covers its import's whole range, and imports are made by chunk around what was bound (import_chunk). A temporary measuring patch counted, per 64 KB block of the live mirrors, whether any guest_pools entry was in it. Of 1.1-1.6 GB of mirrors, 243-533 MB had none (DLAA 2560x1440: Hyrule Field, Kokiri, and up to 533 MB after Castle Town to the field): chunk padding, and pools the game had let go. On an 8 GB card's allowance (6,900 MB) the field walk peaked at 6,952 MB, with 321 frames over 33 ms in five minutes.

Change (RX_VK_MIRROR_SPARSE=1; off by default, and only with strict sparse residency for buffers):

• gpu/vk/sparse_mirror.h: which blocks a game pool is in, block spans, runs, and copies clipped to blocks with memory or cut by ranges. gpu/vk/vk_vma.cpp: a sparse buffer, and its blocks' memory from VMA (counted in its budget). vmaFreeMemory joins the recorder's drain boundaries (vk_recorder.h, vk_work.h).
• mirror_make: a sparse buffer, with memory for the blocks a game pool is in. The page table points blocks with none at guest memory, so the shaders' pointers reach the game's own memory there, as for a pool with no mirror.
• gpu_buffer_bind: a binding that reaches a block with none is served from guest memory for that submission. It's noted as an in-place read, or, if written, as a direct write with its shadow forgotten, and the block is wanted.
• At each submission's start:
  • Blocks wanted, and every block a game pool has come into since (guest_pools_gen), get memory in one batched vkQueueBindSparse.
  • They're filled as a new mirror is: guest memory, then the captures' pages, after a full barrier.
  • A written range served from guest memory where blocks had memory is copied in again.
  • The write-back diff's shadow forgets what was filled.
• Giving memory back:
  • A block that no game pool has been in, and no binding has reached, for 600 submissions gives its memory back.
  • Its page-table entries go back to guest memory at once.
  • Its memory is unbound and freed once the GPU is past it.
  • Without the 600-submission wait, short-lived pools the game frees and makes again at the same addresses churned about 28,000 blocks and 900 binds in one area change.
  • A block wanted again before its memory is freed takes it back without a bind.
• The binds are not waited for on the CPU:
  • Each such wait waited for the whole queue (about 3 ms); 170-390 of them made the load's longest frame 1.0-1.2 s.
  • Instead they signal a timeline semaphore, and every submission of recordings waits for it on the GPU: the batched flush, a submit, and a parallel segment.
  • Memory an unbind let go is freed once the semaphore has passed it.
  • RX_VK_SPARSE_WAIT=1 waits on the CPU, as before.
• Write-backs, and the inheritance of a retired mirror, are clipped to blocks with memory on both sides. Inheritance also leaves a retired mirror's pending refills to guest memory. A freed sparse mirror's buffer and blocks go once the GPU and its binds are past them.

Measured. RTX 4070 SUPER 12 GB (617.42), Windows 11, DLAA 2560x1440, unattended, the same build with and without the switch:

	Off	On
6 GB allowance (6,144 MB, RX_VK_MIRROR_MB=1536), field walk, three pairs: fps	47.7, 51.5, 49.9	57.3, 55.3, 58.2
the same: frames over 25 ms	3,042, 1,840, 2,114	625, 1,160, 408
the same: VRAM peak	7,327, 6,254, 6,444 MB	6,394, 6,200, 6,387 MB
8 GB allowance (6,900 MB), field walk: fps / VRAM peak	59.0-59.1 / 7,028-7,138 MB	59.1-59.3 / 6,676-7,018 MB
Castle Town to the field: fps	54.0, 52.7	55.4, 52.5, 54.8
the same: mirrors with memory	—	1,267-1,311 of 2,128-2,144 MB
the same: the load's longest frame	816-902 ms	866-961 ms (1,197-1,232 with RX_VK_SPARSE_WAIT=1)
Kokiri Forest: VRAM mean	8,215 MB	7,462-7,507 MB

Screenshots of Castle Town, Kokiri and the field look the same on and off. No device was lost and no write-back warning appeared in any run. The rows are in docs/tested-configurations.md.

Two independent code reviews of earlier versions found ten problems in all, each fixed as described above:

• a binding over blocks with and without memory;
• a block given back and bound again under work still running;
• the write-back shadow;
• blocks only pointers reach;
• the budget;
• blocks that bindings, but no pool, still reach;
• refills lost to inheritance;
• unbinds not batched;
• the take-back search;
• a freed mirror's pending blocks.

Tests. tests/test_sparse_mirror.py runs the header: live blocks, an earlier long pool, partial blocks, spans, runs, clipping across both sides' block ends, and cut ranges. It also checks the wiring in vk_memory.cpp, vk_vma.cpp and vk_replay.cpp. The fixtures of test_direct_buffer_writes and test_parallel_init_order get the helpers they now reach. test_readback_part counts the three new reads of guest memory, each noted. The checks that read these files pass, except 18 that fail the same way on plain main on this PC (their compiler wrappers).

Overlap with open PRs. It merges cleanly with #2711 and #2760. With #2618 (a mirror that found no VRAM asked for again) it conflicts in the same lines of mirror_make (where a whole mirror's buffer is made): no conflict in what they do, since a sparse mirror that cannot be filled falls back to that same whole buffer, where #2618's retry would apply. Whichever lands second takes a small merge, which I'll do if it's this one. #1377, #2338 and #2609 conflict with main already.

Not tested. AMD (RADV and Windows), Intel, Linux; a real 6 or 8 GB card; a long session.

🤖 Generated with Claude Code

[Joey-Magnus]

I merged this into current main and ran tests/run_all.py. test_draw_stream_visibility and test_fsr_image_retirement fail here exactly as they do in CI, and they fail because of this change. Both cut submit_recording (and queue_flush_locked, in the fsr one) out of vk_replay.cpp and compile them alone. Those functions now call memory_sparse_wait, which their fixtures don't declare ("use of undeclared identifier 'memory_sparse_wait'"). They need the same bool memory_sparse_wait(VkSemaphoreSubmitInfo*) { return false; } stub you gave parallel_init_order_test.cpp. test_sparse_mirror passes.

It's off by default and gated on sparseResidencyBuffer, residencyNonResidentStrict and a sparse-binding queue, so other tiers aren't exposed. Only the three recording submits wait on the binds' timeline. A sentence on why no other submission (presents, upscalers, texture uploads) can read a newly bound block would help. The 6 and 8 GB rows come from a 12 GB card with a lowered allowance, not real 6/8 GB cards. Your Not tested section says so, but the summary reads as if they were real. Conflicts: #2618 in mirror_make, and #2795 only in docs.