Skip to content

fix(vk): sweep a pile of dead render targets every present, not every tenth - #2484

Draft
bunbeans2-arch wants to merge 1 commit into
NotRexed:mainfrom
bunbeans2-arch:fix/pile-of-dead-targets
Draft

bunbeans2-arch wants to merge 1 commit into
NotRexed:mainfrom
bunbeans2-arch:fix/pile-of-dead-targets

Conversation

@bunbeans2-arch

Copy link
Copy Markdown

Entering one room (the guard room beside Castle Town's gate, about 15 pots) adds about 2 GB of video memory within a second on a 16 GB card and takes the game to 14.5-14.8 GB there. A session that froze in that room has the same jump at the end of its log (#2180). With this change the same room tops out at 12.2-12.6 GB.

Part of #2180

Read this first: the runs below are on v2.1.0 with this change, not on main. main has moved in this pass since (#2198: the compaction on by default, its hold letting the dead targets go first), and that alone may change this room. A build of main means a new shader store on my PC, so I have not run main in the game, with or without the change. The diff is made against main at 6c8c1bf. There gpu/vk/vk_texture.cpp compiles with it (llvm-mingw, the back end's flags from nx2/build.py; that one file, not a whole build), and tests/run_all.py gives the same result with and without it on my PC (below). If you would rather have the numbers on main first, say so and I will bring them when I have a store for it; if you see from the code that #2198 already covers this, close it.

What goes wrong

  • texture_frame returns on nine presents of ten before the pass that lets go of the dead render targets (the frame's scratch targets another target has drawn over).
  • pool_trim gives back 16 images a present, 64 only when the driver's count is over 90% of the allowance.
  • In that room the game makes its scratch targets faster than that: about 1,300 mid-size targets at once, mostly depth targets at 384x384 and at square sizes that change from frame to frame. One run's peak: 1,465 target images, 5,834 MB of them in 6,528 MB of blocks; rendered textures 4,341 MB; the pool held 696 images (1,505 MB) against its 256 MB bound.

The change

  • texture_frame: a regular pass (every tenth present) that finds its ten presents let 64 or more shifted targets go, or made 64 or more target images new, turns a pile on for the next ten presents. While a pile is on, the presents in between run the pass too, as a sweep: dead targets alone, never a live texture, whatever the budget says. The budget's, the hold's and the driver's-limit passes keep their pace (by_hold needs !sweep, and an extra pass is a sweep). RX_VK_PILE_FAST=0: as before.
  • pool_trim: 64 images a present, not 16, while the pool holds more than twice its bound. RX_VK_POOL_DRAIN=0: as before.
  • One log line every 600 presents while extra passes happen: vk: a pile of dead render targets: N extra passes so far, M targets let go in them. Extra passes do not print the textures retired line (it would be about 60 lines a second while a pile lasts); their count is in the pile's line.
  • Nothing changes while no pile is on: the first check is one comparison a present.
  • Both switches follow this file's RX_VK_X=0: as before pattern (RX_VK_DEAD_TARGETS): the fix is on, the switch gives the old behaviour back. They are documented in their comments, as the dead targets' and the pool's other switches are; docs/development.md does not list those.

Numbers

RX 9070 XT 16 GB (driver 0x8001a0 as the log prints it), i9-9900K, 32 GB RAM, Windows 11. v2.1.0 built from source, with and without the change, 3840x2160, frame generation on Auto, default texture budget (6144 MB), RX_VK_PARALLEL=4, scripted input. "Game" is the game's own video memory, "adapter" everything on the card.

The room, peak in MB (game / adapter):

runs
without the change 14,511 / 15,070; 14,612 / 15,178; 14,832 / 15,324; 14,801 / 15,217
with the change 12,615 / 13,032; 12,584 / 13,008; 12,442 / 12,861; 12,218 / 12,804

The runs without the change were ended early, by one of my safety stops or by the script a few seconds after the door, so I have no fps in the room without it at this budget. With it the four runs stayed 298 s, 567 s, 366 s and 388 s in the room (the third and fourth with sword swings only):

with the change run 1 run 2 run 3 run 4
fps in the room, mean of the one-second rows 54.7 48.4 49.4 48.8
lowest one-second row 47.3 37.5 37.4 37.8
gaps of 100 ms or more between presents in play 0 0 2 (234 and 143 ms) 1 (174 ms)
extra passes / targets let go in them 1,701 / 39,430 6,408 / 116,882 4,392 / 69,223 8,226 / 133,315

The 234 ms gap is where four compute pipelines compiled in play.

A walk back and forth over the drawbridge (frame generation 2x, same route, one run each), for what the change costs where no pile lasts:

without with
fps in the mean 46.7 46.9
lowest one-second fps 30.6 27.6
gaps of 100 ms or more between presents 58 (biggest 331 ms) 58 (biggest 318 ms)
peak game / adapter MB 13,844 / 14,347 13,787 / 14,204

Extra passes ran on 1,368 of that walk's 17,192 presents (8.0%).

What this does not show

  • The freeze itself. It has not come back in this room in any run of mine, with or without the change, so this is "Part of", not "Fixes".
  • Anything on main in the game (above).
  • A one-second rise still happens with the change (+2,178 MB at the door in the third run); what changes is where it tops out.
  • One of eleven starts with the change stalled while the save loaded, before play: a gap of 12.4 s between two presents, and the game read no controller for 14.3 s. The other ten loaded as usual (five of them were starts that only loaded the save and stood for 90 s). That run's first extra pass is logged after the stall had begun; the faster pool drain writes no line, so I cannot place it. I could not tie the stall to the change or clear the change of it.
  • NVIDIA and Intel cards, and cards under 16 GB: not run.

Test

python -I -B tests/test_target_image_pool.py: two of its checks named the lines this changes (the tenth-present return, pool_pressed ? 64 : 16) and are updated; a new check keeps the rule: an extra pass only while a pile is on, an extra pass is a sweep and comes before the hold's by_hold, only a regular pass turns a pile on, an extra pass prints no retired line, and both switches are read. 10 of 10 pass with the change; 3 of 10 fail on main without it.

python -I -B tests/run_all.py on Windows, on main at 6c8c1bf with the change: 375 passed, 6 failed, 31 skipped. Plain main: 375 passed, 6 failed, 31 skipped, the same six (tests/test_fpx_sticky.py, tests/test_bake_portability.py, tests/test_vram_budget_row.py, nx2/test_nup_paths.py, nx2/test_nup_read_name.py and the byte-compile). They are my PC's set-up, not this change: the ones I read stop at a file link Windows would not let my user make, and at a fixture my compiler could not build.

The runs above were made with the build that still printed the retired line on extra passes; apart from the move from v2.1.0 to main, that one condition is the only difference between the change as it ran and this diff.

🤖 Generated with Claude Code

… tenth

Entering Castle Town's guard room adds about 2 GB of video memory within a
second on a 16 GB card: texture_frame lets the dead render targets go on
every tenth present only, and pool_trim gives back 16 images a present.

While a regular pass finds a pile (64 or more shifted targets let go in its
ten presents, or 64 or more target images made new), the presents in between
run the pass too, as a sweep: dead targets alone, never a live texture.
pool_trim takes 64 images a present while the pool holds more than twice its
bound. RX_VK_PILE_FAST=0 and RX_VK_POOL_DRAIN=0: as before.

Measured on v2.1.0 with the change, not on main (the pull request says so).

Part of NotRexed#2180

Assisted-by: Claude Fable 5.2
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
@bunbeans2-arch

Copy link
Copy Markdown
Author

One more pair of runs on v2.1.0 (not main): the drawbridge / market walk with frame generation 2x, the same build both times. Time is counted from the first second the game's video memory was over 11,000 MB, which is the area loaded.

  • With the change: the full walk, 1,464 s (71,986 presents). The game's video memory peaked at 14,059 MB. From present 3,200 on (after the start-up): 322 gaps of 100 ms or more between presents, the biggest 606 ms, none of 1 s.
  • With RX_VK_PILE_FAST=0 and RX_VK_POOL_DRAIN=0: ended after 311 s by my stop on free commit (5,995 MB against a 6,000 MB line), in a second where the game's video memory jumped 1,123 MB to 14,605 MB.

That stop is not a RAM leak. In five walks here the process's private bytes follow its dedicated video memory (about 17 GB plus 1.13 to 1.18 MB for each MB of video memory), so free commit drops in the second the video memory peaks.

One run each, so this is a hint, not a proof.

@YakuzaFanA

Copy link
Copy Markdown

Tried this on NVIDIA with an 8 GB card, since the numbers so far are from a 16 GB AMD card. It helps a lot in Castle Town's market for me.

Setup: RTX 5060 Ti 8 GB, driver 610.88, Core Ultra 7 265F, Windows 11, 1080p. Built on main at 497060f with #2482 merged, then the same build with this PR added on top. Same switches both times; this PR is the only difference.

The market's crowd changes a lot (anywhere from 1,000 to 3,400 draws a frame), so I compared fps at the same draw count. Same route each time, Hyrule Field into the market:

draws a frame without #2484 with #2484
1,000-1,499 27.8 55.7
1,500-1,999 20.0 49.1
2,000-2,499 16.8 44.1
2,500-2,999 26.6 36.6

What changed in the log in the market: VRAM was over the driver's limit in 3 of 4 reports before and 1 of 9 after. Rendered textures went from 900-1,019 MB to 694-875 MB, and dead targets left for a regular pass went from about 103 to 25 (41,363 let go in 2,664 extra passes over the session). Live textures evicted for the budget also dropped, from about 51 a pass to 27. It recovers from the entry dip a lot faster, and I didn't see any visual problems.

One more thing I noticed that this PR doesn't touch: on an 8 GB card the target pool is almost never used. It's given back while pressure is over 90% of the allowance, and an 8 GB card sits there nearly all the time (61 of 76 VRAM reports in Hyrule Field said "VRAM short: given back"). In the market that meant about 4,350 target images made new per 600 presents against about 370 reused. Maybe keeping a small pool of the shapes that come back every frame would cut that churn on smaller cards, but I haven't tried it.

(I used Claude to help read the logs; the playing and testing were mine.)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants