Repository navigation
perf(vk): cap the system-heap budget for idle live imports on Linux - #2839
Conversation
ghost
left a comment
ghost
left a comment
There was a problem hiding this comment.
Thanks for this, and for the long-session numbers. A 2 h run that shows the fade from 60+ to 44-48 fps, with the live-import count and the submit costs next to it, is the kind of evidence these memory PRs need. The diff is small and the mechanism is sound. Moving the env reads to env_int and the watermarks to retired_give_back::want is a nice cleanup.
What happens at the cap. It only gives memory back. Nothing stalls, and no allocation is refused. Past 85% of min(budget, 4500 MB), the valve takes the idle unmirrored pools that are done on the GPU, zeroes their page-table entries, and frees their memory once the GPU is past them. They are imported again at their next binding, and that import is the one that already runs in the background. That is the same path 16 GB Linux PCs have run by default since #479, so I don't see a new device-loss route on Linux. What changes is how often it runs, and on which PCs:
- It now runs in steady state, not only under pressure.
usedis the whole non-device-local heap as VMA/the driver reports it, not just live imports. Your own capped run sat atsystem heap 3878 of 15434 MB, right on the line. Wherever the rest of the heap alone is near 3.8 GB,wantstays above zero at every probe, and every import unbound forkImportIdle(600 submissions, under 2 s at 6 submits a frame and 60 fps) goes back. Most of the time that is fine. But the known caveat in the comment applies far more often now: a pool reached only through the page table reads zeros until it is bound again. Before this change, that could only happen at 85% of the full budget. - 16 GB PCs and the Steam Deck change too. On those, the threshold drops from ~85% of a ~7.5 GB budget (~6.4 GB) to 3.8 GB. Integrated GPUs have mirrors off by default (
mirror_on), so on the Deck and the 780M-class APUs every live import is unmirrored and can be taken. I'd expect the most churn there, and nothing in the PR measures it. The Deck is Tier 1 (docs/workflow.md). - New hardware runs the valve. On 32 GB+ Linux PCs on NVIDIA's driver and Intel, this valve has never run by default before. The run here is one RX 9070 XT.
Is 4500 MB justified? Not yet. The run behind the numbers is main with RX_VK_IMPORT_HIGH=25/RX_VK_IMPORT_LOW=20 against the full budget (3858/3086 MB), not this PR's default (3825/3375 MB). The high mark is close, but the low mark is ~300 MB different. The value itself has only been looked at on a 32 GB PC. Under our testing rule (before/after on the same route, hardware named), what I'd need before merging:
- A run of this PR's build with no
RX_VK_IMPORT_*variables, on the same route and for about the same length as the 2 h baseline. Please also say whether the baseline had the sameRX_VK_MIRROR_MB=4096,RX_VK_RECORDER=threadandRX_VK_BATCH=0as the capped run. The second table row doesn't say. - A 16 GB Linux PC or a Steam Deck, before and after: the count of
idle imports given backlines, the fps, and a revisit of an earlier area (any stall while its pools are imported again). If you don't have one, say so and we'll ask a tester. - Ideally one NVIDIA Linux run, since that driver has never had this valve on by default.
Design note. The cost you measured comes from the count and bytes of live imports (amdgpu's BO list, the capture ranges), not from pressure on the system heap. A cap on idle live-import bytes, or a cap on the system-heap figure applied only above 12 GB, would target that more directly and leave 16 GB PCs and the Deck as they are. In this PR's form, at least make the cap tunable (say RX_VK_IMPORT_CAP_MB, with 0 meaning off), so testers can try other values without rebuilding.
Overlap with #2759. That PR retires the imports of pools the game has finalized. On its 32 GB test PC, it took live imports from 257 live (8,720 MB) to 74 live (2,816 MB). That is the same pile-up you measured here, fixed at the source. If most of your 221 live pools are from areas the game has unloaded, #2759 removes them without a cap. I'd land #2759 first and re-measure this PR on top of it. If live imports still grow past ~3 GB over a long session, the cap is clearly earned. #2825 (trimming partly-live mirrored imports) is complementary, and the two shouldn't conflict textually.
Nits:
- Setting
RX_VK_IMPORT_HIGH=85now behaves differently from leaving it unset (full budget vs. capped). That's a reasonable escape hatch, but please say so in the block comment. The current wording ("RX_VK_IMPORT_HIGH= uses the full budget") is easy to miss. Note thatRX_VK_IMPORT_LOWset on its own does not lift the cap. - The config line is logged once with the first probe's budget.
live_budgetis recomputed at every probe, and the driver's budget moves, so the logged cap can go stale. That's harmless; just noting it.
CI hasn't run on this head yet (first-time contributor runs are awaiting approval). There are no merge conflicts with main.
| /// The system-heap cap for idle live imports (imports_give_back) when RX_VK_IMPORT_HIGH is not set: on 32 GB+ Linux PCs | ||
| /// the heap's budget is 15+ GB, and leaving imports_give_back off above 12 GB (or waiting for 85% of 15 GB) let 200+ idle | ||
| /// unmirrored pools stay in the driver's buffer list and the submit-time capture ranges over a 2 h session (7.4 GB live). | ||
| constexpr uint64_t kLiveBudgetCap = 4500ull << 20; |
There was a problem hiding this comment.
Where does 4500 MB come from? The measured run was 25%/20% of the full 15,434 MB budget (3858/3086 MB), not 85%/75% of 4500 (3825/3375 MB). Also, this is a constant on a figure that includes everything else in the system heap. On a 16 GB PC or the Deck, it moves the threshold from ~6.4 GB down to 3.8 GB, which hasn't been measured. An RX_VK_IMPORT_CAP_MB override, or applying the cap only where the budget is >= 12 GB (the PCs that were off before), would keep this change scoped to the case you measured.
There was a problem hiding this comment.
Done in 6c9c7f4: live_budget now leaves budget < 12 GB (16 GB PCs and the Deck) untouched by default and adds RX_VK_IMPORT_CAP_MB (> 0 sets the cap in MB, 0 turns the >= 12 GB cap off). Setting RX_VK_IMPORT_HIGH or RX_VK_IMPORT_LOW without RX_VK_IMPORT_CAP_MB also uses the full budget.
There was a problem hiding this comment.
| if (!small || used <= budget / 100 * (uint64_t)high) return; | ||
| const uint64_t target = budget / 100 * (uint64_t)low; | ||
| const uint64_t want = used > target ? used - target : 0; | ||
| const uint64_t want = retired_give_back::want(used, cap, high, low); |
There was a problem hiding this comment.
used here is the whole non-device-local heap, not just live imports: your run sat at 3878 MB with 2.3-2.7 GB of live imports. Wherever the rest of that heap stays near the cap, want is never 0, and every unmirrored pool idle for 600 submissions (under 2 s) goes back at each probe. That also makes the zero-read caveat in the block comment much more likely than before. Integrated GPUs have no mirrors, so there it applies to every import. It would be good to see the idle imports given back counts from a Deck or 16 GB run.
There was a problem hiding this comment.
Scoped to budget >= 12 GB in 6c9c7f4 so 16 GB PCs and integrated GPUs / the Steam Deck keep their existing budget threshold (~6.4 GB on 16 GB RAM) unless RX_VK_IMPORT_CAP_MB is explicitly set.
There was a problem hiding this comment.
Scoping fixes the 16 GB PCs and the Deck, thanks. On 32 GB+ PCs, used is still the whole heap, so whether the valve now runs all the time there is what the defaults run should show (the count and how often idle imports given back logs). Separately, on 32 GB+ PCs with no mirrors (integrated GPUs, GCN), every import passes this filter: see the review for scoping the default cap to mirror_on().
|
Thanks for the thorough review! I pushed
Regarding #2759 and hardware coverage:
|
ghost
left a comment
ghost
left a comment
There was a problem hiding this comment.
Thanks for the quick turnaround. The scoping is what I hoped for, and the override is useful.
Fixed
- 16 GB PCs and the Deck. Below a 12 GB heap budget,
live_budgetnow returns the heap's own budget whateverRX_VK_IMPORT_HIGH/LOWsay, unlessRX_VK_IMPORT_CAP_MB> 0 is set. So 16 GB PCs (~7.5 GB budget), the Deck and 24 GB PCs keep main's 85%/75% of the full budget. RX_VK_IMPORT_CAP_MB. It goes throughenv_int, so unset, empty or non-numeric falls back to -1 (the default). A negative value also means the default,0turns it off at >= 12 GB, and a value above the budget clamps to it.want()now returns 0 for a zero budget. That all parses safely.- The escape-hatch nit, and the baseline's settings. Both are spelled out now. Thanks.
Still open: a run of this PR's defaults. The reply doesn't cover it, and it's what merging needs (our rule: before/after on the same route, hardware named). Both tested rows are still v2.1.0. The capped one is main with RX_VK_IMPORT_HIGH=25/LOW=20, not this PR's 3825/3375 MB. Your clarification also shows the two rows differ in three more settings: RX_VK_MIRROR_MB=4096 against the default 2048, RX_VK_RECORDER=thread and RX_VK_BATCH=0. The mirror budget changes how many pools lose their mirror and become candidates for this valve. The recorder and batch settings move the submit times and the frame rate. So the step from 44-48 to 63-68 fps can't be credited to the cap yet.
Main has also moved under this since the baseline. Three PRs merged today all shrink live import bytes: #2685 (4 MB import chunks where mirrors are on), #2825 (an import mostly outside the game's pools is trimmed to its live parts) and #2795 (a retired pool with a refused mirror gives its import back). The 221 live / 7.4 GB pile was measured before all three. What I'd like, after merging main:
- Current main and this PR (default
hookshot_env.txt, noRX_VK_IMPORT_*, the other settings the same in both), on the same route, each for about the 2 h of the baseline, on your 9070 XT. Please report thevk: importslive line, theidle imports given backcount and how often it logs, the system-heap figure, capture/queue submit, the readback wait and the fps. - Note anything that drew wrong when you went back to an area: missing terrain or textures, or a hitch while its pools are imported again.
That one A/B answers the 4500 MB question, and the one about the valve firing in steady state (used is the whole heap). If main alone now stays near 2-3 GB live over 2 h, the cap may want to be higher. If it still climbs, 4500 is earned. I wouldn't wait for #2759. It isn't ready, and RX_VK_IMPORT_CAP_MB lets us retune once it lands.
New: PCs with no VRAM mirrors and 32 GB+. The description says the cap is for pools whose mirrors were already given back (MIRROR_NONE, was_mirrored). The idle filter doesn't check was_mirrored, though: it takes any MIRROR_NONE/MIRROR_REFUSED pool. Where mirror_on() is off by default (integrated GPUs, and GCN cards since #2343), every import is unmirrored. On a 780M/890M laptop or an RX 580 PC with 32 GB, the default would now take every import idle for 600 submissions once the heap passes 3.8 GB. The valve was fully off there before, and that is the case furthest from the one you measured, with the page-table zero-read caveat on every pool. I'd apply the default cap only where mirror_on() is true, which is the measured case. RX_VK_IMPORT_CAP_MB > 0 can still turn it on anywhere for testing. NVIDIA and Intel Linux at 32 GB+ stay untested. That's fine to say in the description, and we'll ask testers.
Nits:
- Setting only
RX_VK_IMPORT_LOWon a 32 GB PC now turns the valve on against the full budget. On main, onlyRX_VK_IMPORT_HIGHdid. That's reasonable; one line in the description would do. - See inline: the 12 GB test now runs at every probe.
Merge conflict. It's only docs/tested-configurations.md, because both sides add rows at the top of the table. vk_memory.cpp, the header and the tests merge cleanly with today's main. Please merge main. On this head only the title check has run: the build and test workflows don't start while the PR conflicts, and on the previous head they were waiting for a maintainer's approval. We'll approve them once main is in.
| const uint64_t cap = (uint64_t)env_cap_mb << 20; | ||
| return budget < cap ? budget : cap; | ||
| } | ||
| if (env_pct_set || budget < kSmallBudget) return budget; |
There was a problem hiding this comment.
Nit: main decided "under 12 GB" once, at the first probe (the static small). Here it's evaluated at every probe against the budget the driver reports right then. Where that budget sits near 12 GB, the threshold jumps between 3.8 GB (capped) and ~10 GB (85% of the full budget) from one probe to the next. Deciding the branch once, as main did, would keep it steady, and would match the config line that's logged once.
….3.0 Fewer idle threads spinning (NotRexed#2709), bounded waits for shaders still translating (NotRexed#2586), the first start after a preparation warms the pipeline cache (NotRexed#2865, NotRexed#2869), native float atomics (NotRexed#2306), the 97% squeeze past the allowance (NotRexed#2585), output choices from the screen's modes (NotRexed#2566), FSR 4 twice as fast on RDNA 2 (NotRexed#2798), the import cap (NotRexed#2839) and the final NotRexed#2825 (mirror trims settle). Kept: the fork's play() returning the game for Steam's wait (now through start_scoped's process), the handheld texture caps, the RDNA2 detection. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01C7fdTPXbwDRohew9qanvbp
On Linux PCs with 32 GB+ RAM (
budget >= 12 GB),imports_give_backwas off by default, letting 200+ idle unmirrored pools from earlier areas stay inimportedover a two-hour session (7.4 GB live): on RADV (amdgpu), that doubledqueue submittime (0.55 → 1.14 ms), tripled GPU write-back waits (2.19 → 6.29–7.03 ms), grew the submit-time capture scan from 2.4 GB to 7.2 GB, and faded the frame rate from 60+ fps down to 44–48 fps. This keeps 16 GB PCs and the Steam Deck (budget < 12 GB) unchanged, turnsimports_give_backon forbudget >= 12 GBLinux PCs with a default 4,500 MB cap (~3,825 MB high / ~3,375 MB low), and addsRX_VK_IMPORT_CAP_MB(0= off) so the cap can be tuned or disabled without rebuilding.Part of #993
Part of #1000
What changed
retired_give_back::live_budget(gpu/vk/retired_give_back.h,gpu/vk/vk_memory.cpp).live_budgetreturnsbudget, so 85% / 75% applies to the full ~7.5 GB budget as in vk: imports given back under system-heap pressure on 16 GB PCs #479, unlessRX_VK_IMPORT_CAP_MB > 0is explicitly set).imports_give_backwas off):live_budgetcaps the effective budget at4,500 MBby default (~3,825 MB high / ~3,375 MB low) so unmirrored live pools whose VRAM mirrors were already freed bymirrors_give_back(mstate == MIRROR_NONE,was_mirrored == true) do not stay in RADV's per-submit BO list andcap_rangesfor the whole session.RX_VK_IMPORT_CAP_MB:RX_VK_IMPORT_CAP_MB=<MB>(> 0) sets the cap in MB on any budget so testers can try other values without rebuilding;RX_VK_IMPORT_CAP_MB=0turns the>= 12 GBcap off (restoringmain's behavior on 32 GB+ PCs while keeping< 12 GBPCs unchanged).RX_VK_IMPORT_HIGH/RX_VK_IMPORT_LOW: setting either watermark switch withoutRX_VK_IMPORT_CAP_MBlifts the default 4,500 MB cap and uses the full system-heapbudget(RX_VK_IMPORT_HIGH=0still turns the valve off everywhere). Windows remains off by default (high = 0unlessRX_VK_IMPORT_HIGHis set).imports_give_backreads its switches viaenv_intand sharesretired_give_back::want, covered bytests/test_retired_give_back.py.Tested
Arch Linux (KDE Plasma Wayland, 2560x1440 window,
/dev/udmabufactive), AMD Radeon RX 9070 XT 16 GB (Mesa RADV 26.2.6), Intel Core i5-13600KF, 32 GB DDR4 (15,434 MBsystem heap budget),v2.1.0 (source e561602e), multi-area play across Hyrule:hookshot_log.txt)40,920presents, defaulthookshot_env.txt)684,120presents, defaulthookshot_env.txt,imports_give_backoff)176,640presents,RX_VK_IMPORT_HIGH=25,RX_VK_IMPORT_LOW=20→ ~3,858 / ~3,086 MB;RX_VK_MIRROR_MB=4096,RX_VK_RECORDER=thread,RX_VK_BATCH=0)vk: imports: ... live55 live (2000 MB)221 live (7392 MB)(67 retired given back, 2992 MB)50–64 live (2256–2720 MB)off (budget >= 12 GB)off257 (7872 MB) in all(system heap 3878 of 15434 MB)capture: a submit0.27 ms(scan 0.05 over 734 ranges, 2417 MB)0.50 ms(scan 0.09 over 334 ranges, 7218 MB)0.18–0.43 ms(scan 0.08 over 3923–4251 MB)queue submit0.55 ms1.11–1.14 ms0.32–0.73 msreadbacks of GPU resultswait2.19 ms each(270 of 3000)6.29–7.03 ms each(312–340 of 3000)0.64–1.14 ms each15351 MB (peak 16107)/1511 MBswapratio 1.352, game's frame rate 44)gpu/vk/vk_memory.cppandretired_give_backchecks pass (tests/test_retired_give_back.py,tests/test_texel_buffer_ownership.py,tests/test_import_buffer_binding.py,tests/test_mem_trim.py,tests/test_vram_budget_row.py, etc.).docs/tested-configurations.md.Cost where not measured / caveats:
budget < 12 GB) are unchanged by default (live_budgetreturns the fullbudgetbelow 12 GB).