[corentinbonjour]

Every translated shader stage builds three new IR pools, and the default block chunk (8192 IR::Blocks of about 8 KB) commits 68 MB a stage and then faults in fresh pages. Reusing each thread's emptied pools, with small block chunks, cuts translation CPU by 20 to 30% and peak commit by about 6x, with the same SPIR-V.

What changed

gpu/vk/vk_shader.cpp:

• New struct Pools: insts{8192}, blocks{32}, flow_blocks{32}.
• spare_pools() is a function-local thread_local list of emptied pools.
• Translation holds a unique_ptr<Pools>, taken from the thread's list or made new. Its destructor clears the program, then calls ReleaseContents on flow_blocks, blocks and insts (the order the members were destroyed before), and puts the pools back.
• translate() reads t->pools->... and is otherwise unchanged.

Translations alive at the same time (vs + fs, task + mesh + fs) each get their own set, so a thread keeps at most 3. Nothing is shared between threads. Translation objects are locals of the translating functions, so they are always destroyed on their own thread.

Measurements

Synthetic Maxwell fragment programs (24 programs, 60 to 3200 instructions, varied branching and textures), built with llvm-mingw clang 23 from the v2.0.14 builder at -O2 -march=x86-64-v3 -ffp-contract=off against the repo's Eden and sirit sources. Each iteration keeps two translations alive at once, then emits SPIR-V for both. 9 interleaved repetitions, medians [min-max]:

Threads	Wall (s)	Process CPU (s)	Peak commit
12 x 120	1.155 [1.113-1.555] -> 0.887 [0.829-1.054]	11.64 -> 9.66	1647 -> 273 MB
4 x 120	0.784 -> 0.574	2.99 -> 2.09	559 -> 93 MB
1 x 200		1.078 -> 0.860 (about 0.55 ms a stage)	143 -> 25 MB

Reuse with the default chunk sizes is as fast but does not lower peak commit, and leaves about 140 MB committed per idle thread. Smaller chunks without reuse lower commit but are not faster. Both parts are needed.

This speeds up the translation step only. In a prepare the driver's pipeline compiles dominate, and the whole-prepare effect was not measured on game data.

How it was tested

• The SPIR-V of all 24 programs hashes the same before and after, in every mode and repetition, and the cross-thread check found 0 mismatches.
• New tests/test_translation_pools.py (stdlib Python and a C++ fixture): it cuts Pools, spare_pools and Translation out of vk_shader.cpp and compiles them with Eden's real object_pool.h and stand-in IR types of the real sizes. It checks that no allocation is over 4 MB, that the next translation on a thread reuses the pools without a new large allocation, that two live translations get different pools, that another thread gets its own, and that every IR object is destroyed. It fails on main, fails with the default block chunk, and fails without the push back to the list. It is skipped without a C++ compiler.
• The real vk_shader.cpp compiles with llvm-mingw clang 23 at the build's flags.
• python -I -B tests/run_all.py on Windows, with and without llvm-mingw on PATH: the same failing set as untouched main at 32693f0, none new.

Notes

• Threads keep their emptied pools until they exit. Prepare's workers are joined, so they free them. The persistent background workers keep up to 3 sets each, about 23 MB resident per thread on these synthetic programs; game shaders are probably smaller, not measured.
• vk_shader.cpp is part of the shader stamp, so like any edit to it this makes caches retranslate once.
• perf(prepare): build fewer predicted pipelines, size the pools to the usable CPUs, and keep translated programs across cache-code changes #2323 also edits vk_shader.cpp but not Translation or translate(): its diff applies cleanly on top of this change.

[hckrman101]

Thanks, this is a clean change and the write-up is very good. I went through it with object_pool.h open next to it.

On state leaking between translations: I can't find a way for it to happen. Create() always runs construct_at, so every Inst and Block starts out freshly constructed even when it sits in reused storage. ReleaseContents() runs the same Chunk::Release() that ~Chunk ran before, and the destructor keeps the old teardown order (program, then flow_blocks, blocks, insts). Nothing else in the pool carries over. After a release used_objects is 0 and node points back at the root chunk. The one thing that does change is where the IR objects land in memory, and those addresses were never stable anyway (malloc, ASLR). If some pass depended on them, the output would already differ from run to run. So I agree the SPIR-V should be byte-identical. The 24 synthetic programs show that for real Maxwell code, but they aren't game shaders. If you have a dump, a local before/after of the SPIR-V hashes over a real prepare would settle it: run with an empty cache, then post just the program count and the number of mismatches. Not blocking.

Lifetime: every Translation is a local of translate_into / translate_mesh_into and never leaves its thread. With the destructor declared, Translation has no implicit move, so a null pools can't sneak in. guarded_graphics's siglongjmp only jumps within its own frame, so no Translation is ever skipped. The workers that #2195's stall guard detaches still free their spare list when they exit. A worker stuck in the driver for good was already leaking its stack, so this adds nothing there. The prepare's joined workers free theirs as you describe.

Memory: see the inline note. Each spare set stays at the high-water mark of the largest stage its thread has translated. That's fine, but it's worth knowing for the replay thread and the vk_jobs workers.

Cache stamp: yes, one retranslation, as you say. Since the output is unchanged that only costs time. #2309 also edits the lines just below translate(), so whichever lands second will need a small rebase.

CI: the failing Python checks are test_vcore_fs_open.py / test_vcore_video_acquire.py (env_on undeclared), which this PR doesn't touch. The appimage job failed on DNS while fetching packages. The new test_translation_pools.py passes on Linux and Windows.

Approving. The inline notes are nits.

[hckrman101]

Nit, on memory: when a translation fills the root chunk, ReleaseContents() squashes all of its chunks into one root of the combined size, and later translations never shrink it. So each spare set stays at the high-water mark of the largest stage its thread has seen, for the life of the thread (the replay thread and the detached vk_jobs workers live as long as the game). It's probably small for real shaders, but one huge stage would pin that memory for good. A cheap guard would be to keep only sets whose pools stay under a few MB, and simply drop the rest instead of pushing them back. Up to you.

[corentinbonjour]

Done in 75412d5: the destructor estimates the stage's IR from the program (blocks x (IR::Block + Flow::Block) + instructions x IR::Inst) and, over 8 MB, frees the pools instead of keeping them. It is an estimate, since the pool can hold more than the live IR after dead code is removed, but it catches the one huge stage case. test_translation_pools.py checks that a translation over the limit does not keep its pools, and fails without the check.

[hckrman101]

Nit: push_back in ~Translation could throw bad_alloc from a noexcept destructor. That only matters in theory, but spare.reserve(3) the first time through (task + mesh + fs is the most that are alive at once) would make it impossible.

[corentinbonjour]

Done in 75412d5: the thread's spare list reserves 3 when it is made, so the push in the destructor cannot allocate. The test checks the capacity.