[Joey-Magnus]

When VRAM is tight, the eviction pass throws out the game's cached shadow map, and shadows go missing or come out wrong until the game draws it again. This keeps such textures through evictions forced by the texture budget.

Refs #2209 (dark, moving shadows over Hyrule Field) and #993. Tracking: #995.

What goes wrong

• The game draws its 12288x12288 shadow map (NVN 0x32, 288 MB), and a 4096x4096 one, finalizes them, and keeps sampling them for many frames.
• They are depth images, so the eviction pass can't write their texels back into guest memory before letting them go.
• None of the existing keep rules covers them:
  • keep_finalized only keeps textures of 4 MB or less.
  • The unflushable rule only keeps textures that are not finalized and under 256 MB.
• So under VRAM pressure they are evicted. The game samples them again, they are made again over storage the GPU rendered (so nothing is uploaded), and they are sampled empty until the game draws them again about 40 frames later.

From the log (RX_VK_LOG_REMADE=1, main + this branch's base, Dodongo's Cavern, RX_VK_VRAM_MB=512):

texture #2224 0x1147bf6000 12288x12288x1 x1 NVN 0x32 the GPU drew goes with no copy of its texels (finalized, unused for 403 frames, over the budget)
texture #4806 0x1147bf6000 12288x12288 NVN 0x32, made again 75 frames after its eviction, sampled EMPTY (storage the GPU rendered, not uploaded) at frame 7975

The logs on #2209 (GTX 1660 Ti 6 GB), #2297 (RTX 4060 8 GB) and #2238 (RTX 4060 Laptop 8 GB) show the same 12288x12288 map created again two to four times in one session.

The change

• Texture::resampled is set where a texture the GPU drew is sampled after the game finalized it, which is how a cache the game keeps behaves.
  • Render targets of a single frame are finalized after their last use, so they don't get the flag.
  • It is set on any sample after finalizing, not only "60 frames after drawing". The shadow map is sampled within a few frames of being drawn, then sits unused for about 400 frames before the eviction takes it. A first version that only set it after 60 frames still lost it.
• The eviction pass keeps such a texture unless something has drawn over its storage since, as long as it was used in the last 1,800 presents (keep_resampled::kMaxUnused, the same line after which fix(vk): stop the VRAM compaction's hold from evicting textures in use #2198's reserve lets a long-unused texture go). One from an area left behind goes as before.
• The cap is RX_VK_KEEP_RESAMPLED_MB (512 by default; 0 evicts as before). The candidates' total is counted before the pass, and the pass drops the least recently used beyond the cap, as keep_finalized does, so the cap goes to the most recently used. The rule is in gpu/vk/keep_resampled.h.
• resampled is cleared wherever released goes back to false (the texture is drawn again).
• keep_finalized doesn't count a texture keep_resampled already covers, so none is in both totals. With RX_VK_KEEP_RESAMPLED_MB=0 it keeps them as before.
• The other rules are unchanged.

Numbers

Five-minute headless scripted walks in Dodongo's Cavern (HOS_HEADLESS=1). The texture budget is forced down to 512 MB so the eviction runs constantly, with RX_VK_LOG_REMADE=1. Same save, same build except for this change.

	Without this change	With it
Textures made again and sampled EMPTY	2 (both NVN 0x32 shadow maps)	0
Textures the GPU drew that went with no copy	91	46
kept, not evicted lines from this rule	none	20
fps	60	60
Errors, device lost	none	none

Tested on: Linux (Ubuntu 26.04.1, KDE Plasma 6.6 Wayland), RTX 4070 Ti 12 GB with driver 595.99.02, Ryzen 7 9800X3D, 2560x1440 DLAA. There is a row in docs/tested-configurations.md.

Not tested:

• a real 6-8 GB card (the pressure here comes from RX_VK_VRAM_MB=512)
• AMD, Intel, Windows
• a long session in Hyrule Field

The cost: up to 512 MB of VRAM kept for textures that would otherwise have been evicted under pressure. In these runs that was about 320 MB, the two shadow maps.

Test

python -I -B tests/test_keep_resampled.py:

• checks the rule's cases: kept, drawn over, uploaded rather than drawn, not resampled, the cap and =0
• checks that vk_texture.cpp still asks the rule, sets the flag and reads the switch

The full tests/run_all.py passes: 361 passed, 0 failed, 14 skipped.

🤖 Generated with Claude Code

[hckrman101]

Thanks, the diagnosis is convincing: depth, so not flushable; finalized, so neither keep_finalized (4 MB cap) nor the unflushable rule (!released) covers it, and the remade-over-GPU-storage path then samples it empty. The RX_VK_LOG_REMADE lines and the before/after table make the case well. The rule itself is small, has a switch, and the test covers its cases.

What I'd like changed before this goes in is how the 1 GB is spent, because it matters most on the 6-8 GB cards this is for:

1. Oldest first, no age limit. The eviction loop walks order from least recently used, and keep_resampled::keep hands out the cap first come, first served. So the textures that take the 1 GB are the stalest resampled ones, and once the cap is used up, a younger one (the current area's shadow map) is the one that goes. Nothing ever lets a kept one go except being drawn over, so a shadow map from an area left behind stays for the rest of the session. keep_finalized does this the other way round on purpose (sum first, drop the least recently used beyond the cap), and elsewhere in this loop a course unused for 3600 presents is allowed to go. I'd suggest the same here: count the candidates' total up front and let the oldest go beyond the cap, and/or don't keep one unused for more than N presents (3600 would still cover the 400-frame gap in your log).
2. The cost on 8 GB. The default budget there is roughly 5-5.5 GB, so a 1 GB keep is about a fifth of the cache that the pass can't touch, and everything else is evicted harder to make up for it. Your runs kept ~320 MB (the two maps); a default nearer that (say 384-512 MB) would fix the reported case with less risk. A run on a real 6-8 GB card, or at least RX_VK_VRAM_MB around 4000 in Hyrule Field for a few minutes, would help here, since the 512 MB run with a 12 GB card doesn't show what the driver does near its limit.
3. #2198 (compaction hold, on by default there). Both PRs add lines right after const bool overwritten = shifted || drawn_over(tex);, so whichever merges second has a small textual conflict. The semantic one matters more: #2198's reserve line exists to let go of long-unused textures, and a 400-frames-unused 288 MB shadow map is exactly that. With this rule placed first, the hold can't take it and will take live textures instead to meet its line. Fine if that's the intent, but it should be decided when the two are merged together, and the rule's VRAM should probably be counted by the hold.

Smaller things:

• resampled is never cleared. released goes back to false when the GPU writes the texture again (lines 2613/3132), but resampled stays, so a reused target that was once sampled after finalizing (e.g. last frame's shadow map read before it's drawn) keeps the flag for good. Clearing it alongside released = false would make the flag mean what the comment says.
• A small finalized texture that is also resampled is now caught by this rule first, but it is still counted in finalized_kept, so the keep_finalized total stays inflated and its LRU drops a few more small ones than it needs to. Minor; skipping resampled ones when summing finalized_kept would fix it.

CI: the failing appimage and SPIR-V fixtures jobs fail in the apt/toolchain install step (DNS to archive.ubuntu.com), not in anything this touches; Windows build and the Windows Python checks pass, the Ubuntu ones were still pending.

[hckrman101]

order is oldest first, so this gives the cap to the least recently used resampled textures and evicts the newest ones once it's full; and a kept one never ages out unless drawn over. Could this follow keep_finalized (total first, drop the oldest beyond the cap) and/or skip ones unused for more than ~3600 presents?

[hckrman101]

Fixed in 837ee60: the candidates are summed first, the oldest go beyond the cap, and anything unused for more than 1,800 presents is no longer a candidate. Thanks.

[hckrman101]

1024 MB default: on an 8 GB card that's about a fifth of the texture budget pinned. Your runs needed ~320 MB; a default around 384-512 would cover the shadow maps with less pressure on everything else.

[hckrman101]

Default is 512 MB now, thanks. The run at RX_VK_VRAM_MB ~4000 (or on a 6-8 GB card) is still open; see the review.

[hckrman101]

Nit: nothing clears resampled when the GPU draws the texture again (released = false at 2613/3132). Resetting it there would keep the flag to "sampled after its latest finalize".

[hckrman101]

Fixed: it is cleared at both places now, and the check keeps the two counts equal.

[Joey-Magnus]

Thanks for the review. Rebased on current main (7759a6f), and the points are addressed in 837ee60 and 2c3692c.

1. Oldest first, no age limit. It now works like keep_finalized: the candidates' bytes are summed before the pass, and walking least recently used first, the oldest go until the rest fit, so the cap goes to the most recently used. There's also an age limit. Only a texture used in the last 1,800 presents is a candidate, the same line after which fix(vk): stop the VRAM compaction's hold from evicting textures in use #2198's reserve lets a long-unused texture go, so one from an area left behind is evicted as before. 1,800 still covers the ~400-present gap in the log.
2. Cost on 8 GB. The default is now 512 MB. Not done yet: a run on a real 6-8 GB card, or at RX_VK_VRAM_MB ~4000 in Hyrule Field. I'll add a row to docs/tested-configurations.md when I have it.
3. fix(vk): stop the VRAM compaction's hold from evicting textures in use #2198. With the 1,800-present limit, a shadow map unused that long is no longer kept by this rule, so fix(vk): stop the VRAM compaction's hold from evicting textures in use #2198's hold can take it as intended. A recently used one is still kept ahead of the hold. Counting the kept bytes in the hold's reserve fits better in whichever PR merges second, and I'm happy to do it here if this one goes in after fix(vk): stop the VRAM compaction's hold from evicting textures in use #2198.

Smaller ones:

• resampled is now cleared at both places released goes back to false. The test checks that the two counts match.
• keep_finalized no longer counts what keep_resampled covers. 2c3692c fixes a gap in that: with RX_VK_KEEP_RESAMPLED_MB=0 the exclusion still applied, so a small resampled texture lost the protection it had before. It now applies only while the rule is on.

tests/run_all.py on Linux passes. vk_texture.cpp compiles with nx2/build.py's flags.

[hckrman101]

Thanks for the rework. Looked at the rebased head (2c3692c) against what I reviewed at f34b900.

Fixed:

• Oldest first / no age limit: the candidates are now summed before the pass and the oldest go until the rest fit, so the cap goes to the most recently used, the same way keep_finalized does it. Only textures used in the last 1,800 presents count, so a shadow map from an area you've left goes as before. The test covers the cap order and the clamp.
• Default: 512 MB instead of 1 GB.
• Nits: resampled is now cleared at both places released = false, and the Python check makes sure the two counts stay equal (nice). keep_finalized now leaves resampled textures to this rule, but only while the rule is on (2c3692c).
• #2198: the shared 1,800-present line answers the semantic part well enough for now. Counting the kept bytes in the hold's reserve in whichever PR merges second works for me. The textual conflict after the overwritten line is still there for whoever goes second.

Still open, and the reason this is a comment and not an approval:

• The capped-memory run. This changes how VRAM behaves on the 6-8 GB cards it's meant for, and the only in-game evidence so far is the 512 MB-budget run on a 12 GB card. That run shows the shadow maps are no longer sampled empty, but it doesn't show what happens near the driver's limit, where 512 MB the pass can't touch pushes everything else out harder. A few minutes in Hyrule Field at RX_VK_VRAM_MB around 4000 (or on a real 6-8 GB card), with fps, VRAM and any 'made again' counts against main, would settle it. Also, was the tested-configurations row run before the rework (1 GB cap, no age limit)? If so, please say that in the row, or re-run it with the current version.

New, a small nit inline: past its cap, a small resampled texture no longer falls back to keep_finalized.

CI: the one failure (Python source checks (windows-2022, part 2/2)) is tests/test_native_astc_texture_count.py hitting scanf being deprecated on Windows. That test came from #2288 and main's own regression runs fail the same way, so it's not this PR. Everything else passes. Mergeable.

[hckrman101]

Nit: this skips every resampled texture used in the last 1,800 presents, including the ones keep_resampled::keep then lets go because they're over its cap. So a small (<= 4 MB) resampled texture that keep_finalized used to keep goes once the resampled total is over 512 MB. That only matters past the cap and the gap is small. If you want to close it, only skip it here when this rule actually kept it, e.g. by having the pass try keep_finalized after keep says no.

[Joey-Magnus]

Closed in ed765d4: past this rule's cap, a small finalized one falls back to keep_finalized again (finalized_after_resampled). It joins keep_finalized's total at that point in the walk, so the older ones judged before it were judged without it and keep slightly more; I took that over a second pre-pass. Only while the rule is on: with RX_VK_KEEP_RESAMPLED_MB=0 the pre-pass already counts it, and adding it again would double it.

[Joey-Magnus]

Thanks for the second pass. Merged current main in (ed765d4, no force-push), which clears the conflict with #2198.

• fix(vk): stop the VRAM compaction's hold from evicting textures in use #2198, now on by default: this rule still comes before the hold. Its age limit is now vram_compact::kReserveUnused itself, and strict (unused < 1800; before it was <= 1800, so at exactly 1,800 both claimed the texture). A texture this rule keeps is never one the reserve may take, and the two can't drift apart. For textures unused 600-1,800 presents the rule still goes ahead of the hold's own line, so the hold takes other live ones there. That's the trade-off you named, bounded by the 512 MB cap.
• The nit: fixed, see the inline reply.
• tested-configurations: yes, that row was the first version. It now says so (1 GB cap, oldest first, no age limit).
• tests/run_all.py on Linux: 360 passed, 0 failed. The renderer builds with nx2/build.py.

Still open: the run at RX_VK_VRAM_MB ~4000 in Hyrule Field against main. I'll post it here with a new tested-configurations row once the machine is free for it.

[hckrman101]

Thanks. Looked at ed765d4 against 2c3692c. The PR's own diff is still the seven files, and the merge of main brings in #2198 cleanly.

Fixed or answered:

• The keep_finalized gap past the cap (my last nit): closed. A candidate keep lets go now falls back to keep_finalized through finalized_after_resampled, only while the rule is on, so with =0 nothing is counted twice. Joining the total mid-walk, so the older ones keep a little more, is a fair trade against a second pre-pass.
• #2198: using vram_compact::kReserveUnused itself as the age line, strictly (< 1800), means a texture this rule keeps is never one the reserve may take, and the two can't drift apart. That's better than counting the bytes in the reserve. For textures unused 600-1,800 presents the rule still goes before the hold's own line, so the hold takes other live textures there. You named that trade-off and the 512 MB cap bounds it. Fine with me.
• tested-configurations row: it now says it was the first version (1 GB cap, oldest first, no age limit). Thanks.

Still open, and still why this is a comment and not an approval: the run at RX_VK_VRAM_MB ~4000 in Hyrule Field (or on a real 6-8 GB card) against main, with fps, VRAM and the 'made again' counts. You said it's coming. Once it's posted and shows nothing worse than main, I'm happy to approve.

Two small nits inline: a comment in keep_resampled.h got mangled in the edit, and the new pre-pass could check the cheap flags before the overlap scans.

CI: Python checks (Ubuntu and Windows), the Windows Vulkan build and the SPIR-V fixtures pass. The appimage and Hookshot Builder package and setup failures are both actions/upload-artifact getting HTML back from GitHub after the build succeeded, and Conventional Commit title stopped with no error on a valid title. All three look like GitHub hiccups, so a re-run should clear them. Mergeable, no conflicts.

[hckrman101]

Nit: lines 17-20 read like two versions of the comment got merged. "...went unused 400 presents before the eviction took it" has no full stop and runs into "the compaction's reserve's own line, so...". Something like: "Only while used in the last kMaxUnused presents (unused for fewer), the compaction reserve's own line (#2198): one from an area left behind goes as before, and a texture this rule keeps is never one the reserve may take. The 12288 x 12288 map went unused 400 presents before the eviction took it."

[Joey-Magnus]

Fixed in 3fbad81 with your wording: one comment now, ending at "...before the eviction took it."

[hckrman101]

Nit: this runs drawn_over_shifted and drawn_over for every GPU-drawn texture on each pass (every 10 presents), and then drops most of them on resampled. Testing tex->resampled (and the age) first would skip the overlap scans for nearly all of them, e.g. if (tex->resampled && keep_resampled::candidate(...)). It gives the same result.

[Joey-Magnus]

Done in 3fbad81: the pre-pass tests tex->resampled, tex->gpu_written and the age first, and only then runs drawn_over_shifted / drawn_over through candidate. Same result; tests/test_keep_resampled.py still passes.

[Joey-Magnus]

Here's the capped-memory run you asked for, plus the two nits (3fbad81) and a tested-configurations row (229e0e5).

Short version: in Hyrule Field this change does no worse than main, at any budget I tried and with the card full. Frame time, evictions and GPU memory match within noise, and neither side sampled a texture EMPTY. One caveat: in the clean runs the rule itself never had to keep anything (details below).

Setup

• Hardware: RTX 4070 Ti 12 GB, driver 595.99.02, Ryzen 7 9800X3D, Linux.
• One game folder built from this branch at 3fbad81. Main is the renderer of 6c8c1bfb (the main this branch last merged), built the same way and swapped into that folder, so both sides share the host and the flags. Runs alternate main / this.
• Each run: a private kwin_wayland --virtual at 2560x1440, HOS_FPS_MAX=240, RX_VK_LOG_REMADE=1. Load the test save, teleport to Hyrule Field, then 3 minutes of scripted running into the field, turning now and then.
• Frame time is the gap between presents, from RX_VK_FRAME_LOG. GPU memory is the whole card, from nvidia-smi every 2 s. Evictions are the "textures retired" lines' budget N.
• 6-8 GB card: a small CUDA process held 3.8 GB of VRAM for the whole run. That left the card full (11.4-11.5 GB of 12.3 GB in use), and the renderer cut its textures to about 1.4 GB on its own, under the 4000 MB cap.
• Nothing else ran on the PC in the runs below; a check every 10 s confirmed it.

Results

Setting	Run	Median / p99 frame (ms)	Frames > 50 ms	Evicted over the budget	Gone with no copy	Sampled EMPTY	GPU memory, median (MB)
4000 MB, 3.8 GB held (an 8 GB card, full)	main	11.3 / 14.8	5	251	2	0	11497
	this	11.6 / 15.4	4	250	4	0	11446
	main	11.3 / 15.0	5	249	2	0	11450
	this	10.4 / 13.7	0	256	4	0	11427
RX_VK_VRAM_MB=2000	main	10.3 / 13.3	0	174	2	0	8068
	this	10.3 / 13.8	0	173	2	0	8039
RX_VK_VRAM_MB=1000	main	10.3 / 13.8	0	251	4	0	8035
	this	10.4 / 13.9	0	256	6	0	7952
	main	10.4 / 14.0	0	249	2	0	7706
	this	10.5 / 14.5	0	248	6	0	7874
RX_VK_VRAM_MB=4000 (3 pairs)	main	10.2-10.3 / 13.3-13.5	0	0	0	0	8016-8087
	this	10.3 / 13.2-13.9	0	0	0	0	7924-8032

• At 4000 MB without the held memory, the textures stay around 2 GB on this route, so the budget never evicts anything. That's why I added the 2000 and 1000 MB budgets and the held-memory runs.
• "Gone with no copy" is 2-6 on both sides: GPU-drawn textures evicted with no copy of their texels, from feat(vk): say why each texture is evicted, and which GPU-drawn ones go with no copy of their texels #1879's tally.

Caveat: where the rule acted

In the clean runs above, the rule never logged keeping a texture, so they show the 512 MB it reserves doesn't push other textures out harder near the limit, but they don't exercise a kept shadow map under pressure. It kept textures (20 "kept, not evicted" lines, the log's limit) in two earlier runs that went further into the field, with 3.0-3.7 GB of textures and over 3,000 evictions over the budget.

• One of those, a run of this change, was slow: 17 ms median, 97 frames over 50 ms.
• Its main partner with the same pressure (3,362 evictions) ran at 9.9 ms.
• But that run overlapped a full game build on the same PC, and the other one overlapped another game, so I've left both out.
• I couldn't reproduce that pressure cleanly: the route doesn't reliably get that far.

If you want that case covered before approving, a cleaner way to force it would help, say a spot where the cached shadow map is reliably evicted over the budget, like Dodongo's Cavern at 512 MB from the first row.

CI: appimage and Hookshot Builder package and setup (the upload-artifact HTML replies) and the cancelled Conventional Commit title need a re-run. GitHub refuses a re-run from my side, likely because this comes from a fork.