Repository navigation
fix(vk): stop the VRAM compaction's hold from evicting textures in use - #2198
Conversation
…ly long-unused textures With RX_VK_COMPACT=1 the texture budget holds the driver's count at 85% of what it allows by letting go of the excess. Two things went wrong whenever the count stayed over that line: - A pass over its budget takes the oldest textures first and stopped at the first one past the excess. The render targets another has drawn over (the frame's scratch targets) are the youngest, so it never reached them: the sweep that takes them only runs under the budget. Live textures went in their place and the dead targets piled up. - What went had gone unused for 60 presents, so every texture out of view for a second went and came back when the camera turned. The hold's pass now counts what the dead targets hold first and lets all of them go, and takes live textures only for the excess left after that and only ones unused for 600 presents. Passes the other rules ask for (the 90% and 97% rules, the budget itself) are as they were. Assisted-by: Claude Mythos 5.2 Co-Authored-By: Claude Mythos 5.2 <noreply@anthropic.com>
…ng-unused textures With one line the texture cache sat at it, and whatever came in next (a new stretch of the field, a spawn, a recorder started beside the game) went over it. The hold now has a second line five points of the allowance lower (80 for RX_VK_COMPACT_PCT=85): over it, textures unused for 1800 presents are let go, down to that line. The hold's own line still takes those unused for 600, and the 90% and 97% rules are as they were. In a five-minute walk across Hyrule Field at DLAA with 768 MB of the card held by a test allocation, the driver's count was over 85% of the allowance for 47-68% of the time and over 87% for 7-26% (peaks of 91-93%). With the reserve: over 85% for 5%, over 87% for 0-1%, peaks of 87-88%, no more uploads and no more compaction runs. Assisted-by: Claude Mythos 5.2 Co-Authored-By: Claude Mythos 5.2 <noreply@anthropic.com>
…ainst main (RTX 4070 SUPER, DLAA, Hyrule Field) Assisted-by: Claude Mythos 5.2 Co-Authored-By: Claude Mythos 5.2 <noreply@anthropic.com>
|
Tested on Linux, which the PR lists as untested. It holds up there too. Setup:
Results:
🤖 Generated with Claude Code |
|
I tested this on Windows with an RTX 5080 (16 GB), using the same test as Joey-Magnus above. It holds up here too. Setup:
(1% low here is the average frame rate of the slowest 1% of frames. Uploaded is the sum of the trace's Results:
🤖 Generated with Claude Code |
|
Tested this on an 8 GB AMD card, where the hold matters most, and it removes most of the camera-turn hitches with Setup
Method
With the PR, textures made settled at 0.4–0.8 a frame, so the churn described above did not show up. Half of the remaining frames over 50 ms came in one burst on the first camera turn, as new views streamed in. No device loss, validation or "not made" lines in either run. A normal play session afterwards showed no visual problems. Caveats
|
|
Hi! I've been troubleshooting the persistent FPS degradation on an RTX 3070 8 GB and wanted to share my results, as I think they might be relevant to this PR. System:
The issue: The game can run smoothly at approximately 50–60 FPS while exploring Hyrule Field or Kakariko Village. However, transitioning between these areas can cause a significant, persistent performance drop. For example:
I've also compared two recordings in the same location using essentially the same route. The main difference was whether I'd travelled through Kakariko beforehand. The session that had gone through Kakariko performed significantly worse. This makes me suspect something is remaining allocated or otherwise affecting performance after an area transition. What I've tested: Initially, I experimented with the following settings:
These helped somewhat, but the persistent degradation still occurred. I also experimented with Enabling With compaction enabled alongside the earlier settings, the game became noticeably smoother. Areas that previously struggled could run at approximately 50–60 FPS again, and performance after transitions initially seemed substantially better. I then tested Initially, this also performed very well. However, during a longer session, I experienced another prolonged FPS collapse. Performance eventually recovered, but there were periods around 12–21 FPS. The longer log (
The reported allocation sometimes reached approximately 7.6 GB against an allowance of about 7.1 GB. Compaction was actively running and returning memory to the driver, but it wasn't consistently preventing these episodes. Why I'm posting here: I've read the description of this PR and noticed the problem involving textures being evicted after only 60 presents, then repeatedly reloaded when needed again. I'm wondering whether that could explain why performance sometimes remains degraded after travelling between Kakariko and Hyrule Field. I understand that the current fix has been tested with camera movement within an area, but I thought the loading-screen transition behaviour might also be worth investigating. I've attached logs from the different configurations, including one where compaction substantially helped and a longer run where performance degraded again. I'd be happy to test the proposed fix on my RTX 3070, particularly with repeated Kakariko ↔ Hyrule Field transitions, if a suitable build becomes available. Thanks for the work on this project — the improvements over the last few days have already been crazyyy |
hckrman101
left a comment
hckrman101
left a comment
There was a problem hiding this comment.
Thanks, this is a clear diagnosis and the fix for the hold's pass reads right. Counting the dead render targets first and then not stopping at the first texture past the excess fixes the starvation you describe. The prepass uses the same conditions as the loop (need_dead, last_value > completed, shifted/drawn_over), so hold_dead is exactly what the loop then lets go: no dead target is counted and then kept. The fence check (tex->last_value > completed) still comes before any texture is chosen, so nothing the GPU may still read gets evicted. The header is plain integer math and builds the same with llvm-mingw and gcc/clang. Joey-Magnus (Linux, 4070 Ti), nullifyshot (Windows, 5080) and Tempus3214 (RX 6600 8 GB) all confirm it on hardware you didn't have. CI is green.
One thing I'd like settled before merge. The description says passes from the other rules (the budget, 90%, 97%) are unchanged, and the test says the driver's limit clears both lines. In practice that clearing almost never happens. The 90% rule and the reserve share the same pressure, so the reserve's excess is always at least the 90% rule's plus 10% of the allowance. Whenever the 90% rule fires under RX_VK_COMPACT, reserve < budget holds and the pass becomes a hold pass. The 97% rule only wins (budget < evict_to) when reclaim is over ~17% of the allowance. So past 90% and even past 97%, a live texture now has to be unused for 600 presents before it can go, where main let it go after min_unused (60). That is exactly the situation of the 8 GB reports: your own Limits note (93-96% stretches, nothing left to let go) and buasfe's log with the count over the allowance 17 times on a 3070. Paging past the allowance costs more than re-uploads. I'd keep the 600/1800 thresholds for the part of the excess that only the 85/80 lines ask for, and let textures past need_unused go for the part the 90%/97% rules ask for. For example, record the budget those rules set before the hold applies (rule_to) and give hold_target a third tier for it, so that rule's excess is served as before. A test case for "count over 90%: a texture unused 100 presents still goes" would pin it.
Nits:
- buasfe's question (Kakariko <-> Hyrule Field transitions on an 8 GB 3070) isn't answered yet. A test build for them would cover the scene change and the small real card, both listed as untested.
- The
if (budget < evict_to) evict_to = hold_to = 0;comment ("the driver's limit asks for more") reads as if this is the usual path near the limit. A word that it rarely fires once the reserve is set would help the next reader.
| // (and a reserve under that line, for the textures long unused: vram_compact.h. The pass below takes the others only | ||
| // for the hold's own line) | ||
| const uint64_t reserve = vram_compact::held_budget(budget, total_bytes, used, reclaim, allowed, vram_compact::reserve_pct(pct)); | ||
| if (reserve < budget) budget = evict_to = reserve; |
There was a problem hiding this comment.
The reserve's excess is always at least the 90% rule's (same pressure, a line 10 points lower), so whenever the 90% rule set budget above, this replaces it, and the pass that follows protects live textures for 600 presents instead of min_unused. See the review body.
| budget = std::min(budget, total_bytes > over ? total_bytes - over : 0); | ||
| } | ||
| if (budget < evict_to) evict_to = 0; // (hookshot, RX_VK_COMPACT: the driver's limit asks for more: 7/8 of that, as before) | ||
| if (budget < evict_to) evict_to = hold_to = 0; // (hookshot, RX_VK_COMPACT: the driver's limit asks for more: 7/8 of that, as before) |
There was a problem hiding this comment.
This only fires when the 97% rule's excess (by used) beats the reserve's (by used - reclaim, 17 points lower), which takes reclaim > ~17% of the allowance. So over 97% the hold pass, with its 600/1800-present thresholds, usually still decides. Unless that's intended, the driver-limit case needs its own target (review body).
| // (hookshot, RX_VK_COMPACT: the hold's pass, above. A live texture stays once the excess over its line is covered: the | ||
| // reserve's line for one long unused, the hold's own for one unused ten seconds; one used since then stays) | ||
| if (by_hold && !overwritten) { | ||
| const uint64_t to = vram_compact::hold_target(frame_no - last, hold_to, evict_to); |
There was a problem hiding this comment.
Suggestion: a third tier here, where a texture unused for need_unused may go while total_bytes - going is above what the 90%/97%/size rules alone asked for. Then the hold only changes what its own lines take.
There was a problem hiding this comment.
Done in 2a407e1: rule_target / the need_unused tier in hold_target, with the worked 94% case in the test. Resolved from my side.
It was opt-in while its hold could let go of textures in use. With the hold taking the dead render targets first and live textures only when long unused, it is on unless RX_VK_COMPACT=0. The rule is vram_compact::enabled, which tests/test_vram_compact.py runs: not set is on, 0 or empty is off. Assisted-by: Claude Mythos 5.2 Co-Authored-By: Claude Mythos 5.2 <noreply@anthropic.com>
…VK_COMPACT=0 (RTX 4070 SUPER, DLAA, Hyrule Field) Assisted-by: Claude Mythos 5.2 Co-Authored-By: Claude Mythos 5.2 <noreply@anthropic.com>
hckrman101
left a comment
hckrman101
left a comment
There was a problem hiding this comment.
Thanks for the follow-up. Since my last review the branch picked up a merge of main, the switch to on-by-default (vram_compact::enabled, with tests for unset / 1 / 0 / empty) and the new tested-configurations rows. The default flip itself is clean: enabled() matches the docs and the test, RX_VK_COMPACT=0 is a clear way back, and the on-vs-off walk numbers (633/408 frames over 25 ms down to 66/36, shadows never given back) are a good case for it.
Still open, and it matters more now:
- The 90%/97% rules' passes still go through the hold's thresholds. The budget code (3817-3843) and
hold_targetare unchanged, so what I described last time still holds: whenever the 90% rule fires, the 80% reserve sets a lower budget from the samepressureand the pass becomes a hold pass, where a live texture has to be unused 600 presents (notmin_unused, 60) before it can go for the hold's line, and 1,800 for the reserve's. The 97% rule only takes over whenreclaimis above ~17% of the allowance. Before this push that only affected people who setRX_VK_COMPACT=1. With the default flipped it affects every 8 GB card, which is where the count really sits past 90% (your own Limits note on 93-96% stretches, buasfe's 3070 log with 17 "over the driver's allowance" lines). I'd still like the third tier (record what the 90%/97%/size rules asked for before the hold applies, and let textures pastneed_unusedgo for that part of the excess) plus a test like "count over 90%: a texture unused 100 presents still goes" before this ships on by default. If you think 600 presents is right even past 90%, say why and I'm happy to look at numbers, but I'd want them from a real 8 GB card, not the simulated allowance.
Nits, both still open:
- buasfe's question (Kakariko <-> Hyrule Field transitions on an 8 GB 3070) still has no reply. With the default on, a test build for them is the easiest way to cover the scene change on a small real card.
- The comment on
if (budget < evict_to) evict_to = hold_to = 0;(3843) is unchanged; it still reads as the usual path near the limit, when it rarely fires once the reserve is set.
CI: everything passes except appimage, which died on a Launchpad connection reset (RemoteDisconnected) while installing packages, not on the build. A re-run should clear it. Mergeable, no conflicts.
| bool compact_on() { // (hookshot, RX_VK_COMPACT=1: the textures in a pool of their own, compacted: compact_step) | ||
| static const bool v = getenv("RX_VK_COMPACT") && atoi(getenv("RX_VK_COMPACT")) != 0; | ||
| bool compact_on() { // (hookshot: the textures in a pool of their own, compacted: compact_step. On unless RX_VK_COMPACT=0) | ||
| static const bool v = vram_compact::enabled(getenv("RX_VK_COMPACT")); |
There was a problem hiding this comment.
With this on by default, the 90%/97% point from my last review (the hold's 600/1,800-present thresholds deciding passes the 90%/97% rules asked for) now applies to every small card without the switch set. I'd land the third tier before the default flips, or flip the default in a follow-up once that's in.
There was a problem hiding this comment.
With the third tier in, the 90%/97% passes again let textures unused 60 presents go for their part, so the default flip is fine by me.
|
Heads-up: this conflicts with main now. Could you rebase onto main and resolve the conflicts? The point in my last review (the 90%/97% rules' passes still using the hold's 600/1800-present thresholds, now on by default for every 8 GB card) is still open as well. Thanks! |
…p, 1,800-present age limit, 512 MB default - The candidates' bytes are summed before the pass; walking least recently used first, the oldest go until the rest fit, so the cap is spent on the most recently used (as keep_finalized does). - Only one used in the last 1,800 presents is a candidate: the threshold after which NotRexed#2198's reserve lets a long-unused texture go. - RX_VK_KEEP_RESAMPLED_MB defaults to 512. - resampled is cleared wherever released goes back to false (the texture is drawn again). - keep_finalized leaves out what keep_resampled counts, so no texture is in both sums. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
# Conflicts: # docs/tested-configurations.md
… the 90%/97%/size rules ask Review of NotRexed#2198. The reserve's line is ten points under the 90% rule's and counts the same pressure, so whenever the 90% rule set the budget the reserve set a lower one and the pass became the hold's, where a live texture has to be unused 600 presents for the hold's line and 1,800 for the reserve's. The 97% rule only took the pass back with more than ~17% of the allowance free in the textures' pool. So past 90% and past 97% of the allowance nothing used in the last ten seconds went, where main lets go of textures unused for 60 presents. With the compaction on by default that is every card whose count sits there. The budget the size, the 90% rule and the 97% rule ask for on their own is now kept apart from the hold's (rule_budget), and hold_target has a third tier: a texture unused for the eviction's own time (need_unused) may go until the textures are down to 7/8 of that budget (vram_compact::rule_target), which is what a pass over the budget cuts to with the compaction off. The hold's 600 and 1,800 presents only decide what the 85% and 80% lines add to that. tests/test_vram_compact.py: the three tiers, with "the count over 90%: a texture unused 100 presents still goes" and a worked case at 94% of a 7,000 MB allowance; source checks that rule_budget is taken before the hold's lines and joined by the 97% rule's. The comment on the line that clears the hold for the driver's limit says how rarely it fires. Not run in the game: the rules' arithmetic and the source checks only. Assisted-by: Claude Mythos 5.2 Co-Authored-By: Claude Mythos 5.2 <noreply@anthropic.com>
|
Thanks for the review. You were right, and it mattered more once the default flipped. Pushed:
Not run in the game: this push is the rules' arithmetic and the source checks only, and I still have no real 8 GB card. The AppImage failure was a Launchpad connection reset; this push should re-run it. @buasfe sorry for the slow answer. Your log (the count over the allowance 17 times on the 3070) is the case this push is for: past 90% the cache can again let go of textures unused for a second. I can't attach a build myself, but if you're able to build from this branch, a few Kakariko ↔ Hyrule Field transitions on your card would cover the two things nobody has tested here, a scene change and a real 8 GB card. 🤖 Generated with Claude Code |
hckrman101
left a comment
hckrman101
left a comment
There was a problem hiding this comment.
Thanks, this is what I was after. Since my last review: I merged main into the branch (40d8e17) to clear the conflict; the only conflict was the docs/development.md table, where I kept your RX_VK_COMPACT row and main's new RX_VK_REACTIVE row. Nothing else in the merge differs from what main brought in.
Earlier points:
- 90%/97% passes going through the hold's thresholds: fixed (2a407e1).
rule_budgetis taken after the size and 90% rules and before the hold's lines, the 97% rule joins it, andhold_targetnow gives a texture unused forneed_unusedthe rules' part (rule_target, 7/8 of their budget, the same cut a pass makes with the compaction off), then the hold's line at 600 and the reserve's at 1,800, taking the lowest that applies. The dead render targets still count first throughhold_met, and the fence andkeep_rendered/ finalized checks still come after, so nothing in use by the GPU is newly exposed. The 94%-of-7,000 MB case in the test works the numbers through, including "unused 100 presents still goes" and stopping once the rules' part is covered. With this, the description's "passes the other rules ask for are as they were" is true again. - The comment on
if (budget < evict_to) evict_to = hold_to = 0;: fixed, it now says it's rare once the reserve is set and why. - buasfe's Kakariko <-> Hyrule Field question: answered. A run of this head on their 3070 would still be the best check, since the third tier is source checks and arithmetic only so far and no real 8 GB card has run it (Tempus3214's RX 6600 run was on the earlier version).
Overlaps: #1879 landed on main right after the merge; a trial 3-way merge of vk_texture.cpp is clean (it only adds overwritten to struct Evict / evict.push_back and the retire tally), and GitHub shows no conflict. #2368 inserts its keep check just above the hold check in the same loop, so whichever lands second resolves that spot and should make the hold's pass count the kept bytes, as agreed there.
CI: all green except Python source checks (windows-2022, part 1/2), which fails tests/test_native_astc_texture_count.py (its fixture's scanf under -Werror on Windows). Main fails the same test since #2288, so it isn't this PR.
Approving.
…he second review - Conflicts with NotRexed#2198 (compaction on by default): this rule still comes before the hold. - The age limit is vram_compact::kReserveUnused itself and strict (unused < 1,800): a texture this rule keeps is never one the compaction's reserve may take; before, both claimed 1,800. - Past this rule's cap, a small finalized resampled texture falls back to keep_finalized, as before the rule (joins its total at that point). Only while the rule is on: with it off the pre-pass already counts it. - tested-configurations: the row says it measured the first version (1 GB, oldest first, no age limit). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
With
RX_VK_COMPACT=1on a card with little room, textures that were still in use were thrown out and loaded again every time the camera turned, and at some places the game could freeze for a few seconds. Both came from the rule that holds VRAM at 85%; this fixes that rule and gives it a small reserve. With that fixed, the compaction is now on by default (RX_VK_COMPACT=0turns it off), as hckrman101 suggested in Discord.Tracking: #995. Follow-up to #1821 and to @hckrman101's 6 GB result there (4-5x the frames over 33 ms with the switch on).
What went wrong
The hold sets the texture budget to "what the textures hold, less the excess over 85% of the allowance". Whenever the count stays over that line (a small card, a burst of new textures, another program taking VRAM), every pass is a pass over the budget, and two things follow:
The change
vram_compact::hold_met,hold_keeps,kHoldUnused).reserve_pct,hold_target,kReserveUnused). With one line the cache sat at it and whatever came in next went over.RX_VK_COMPACT=0(vram_compact::enabled: not set is on,0or empty is off). It was opt-in while the hold could let go of textures in use.Numbers
Built from this branch (v2.0.11-782-gf0c6e73) and from main (v2.0.11-780-g3932421). Unattended runs with scripted input, DLAA 2560x1440, RTX 4070 SUPER (617.42), Ryzen 9 9950X3D, Windows 11, nothing else running, two runs each. Frames from
RX_VK_FRAME_TRACE.RX_VK_TEST_ALLOWED_MBstands in for a smaller card or for other programs.Camera turning in place in Hyrule Field,
RX_VK_COMPACT=1 RX_VK_TEST_ALLOWED_MB=7000, 50 seconds:Five-minute walk across Hyrule Field,
RX_VK_COMPACT=1 RX_VK_ASTC_BC_ALL=1:No validation, device-lost or "not made" line in any run.
Those runs were on main as it was yesterday (3932421). After rebasing onto today's main (4d68bc8) I ran each once more, built from this branch (v2.1.0-rc.3-50-g3acf477a) and from main (v2.1.0-rc.3-48-g4d68bc83), counted from the end of the load:
In that spin run the branch's count passed the allowance once while the save loaded (7,016 of 7,000 MB), which is what gave the shadows back; main's run did not, and neither did the four spin runs on the earlier base. One run, so I can't say whether it is the branch or chance.
On by default against off
Built from this branch after the default changed (v2.1.0-rc.3-58-g06ce5785, main 49f75dc merged in). No switch set against
RX_VK_COMPACT=0, DLAA 2560x1440, the same card, nothing else running, counted from the end of the load.Five-minute walk across Hyrule Field, two runs each:
RX_VK_COMPACT=0Camera turning in place with
RX_VK_TEST_ALLOWED_MB=7000, one run each: on 60.0 fps, 1,203 MB uploaded, 1 frame over 25 ms; off 59.9 fps, 2,660 MB, 8 frames. No validation, device-lost or "not made" line in any of these.On my local build (the same two commits on an older base, with a measuring hook that holds 768 MB of the card), the reserve alone took the share of a walk spent over 85% of the allowance from 47-68% to 2-5%, and over 87% from 7-26% to 0-1%, with uploads and frame times unchanged.
Limits
RX_VK_COMPACT=1.RX_VK_COMPACT=0is the way back there.Test
python -I -B tests/test_vram_compact.py: the rules' arithmetic (hold_keeps,hold_met,reserve_pct,hold_target), and source checks that the hold's pass counts the dead targets before it chooses, does not stop at the excess, judges a live texture only once it knows whether a target drew over it, and that the driver's limit clears both lines; andenabled: not set is on,0or empty is off. Fails on main.🤖 Generated with Claude Code