Repository navigation
feat(dlss): NVIDIA DLSS Frame Generation through Streamline (Vulkan, manual hooking, Reflex) - #1944
j64esatxzdrhfg wants to merge 1 commit into
Conversation
|
Tested on an RTX 4080 SUPER (so RTX 40 now has a data point). With DLSS Quality it works well: the frames double, all of them reach the screen, and pacing is the most even of the frame generation options we have. The cost is small: the game's own frame rate (not counting made frames) went from 53.4 to 51.2 fps. Two bugs got in the way first, both small:
MeasurementsMarket gate, a scripted 20 s camera spin, 4K output (3840x2160), HDR10, windowed (1920x1080 window, "Composed: Flip"), 239 Hz display, MAILBOX. RTX 4080 SUPER, driver 617.42, Windows 11. A build of main (907ec4e) with this PR merged. PresentMon 2.6 captured display times for every present. One run each:
The log agrees with PresentMon:
Bug 1: DLSS needs the runtime beside hookshot.exe while Streamline is loadedWithout Streamline, NGX loads
I haven't confirmed why. My guess is that Streamline's own NGX start comes first, so the path list Bug 2: RX_VK_DLSSG doesn't turn DLSS-G on
Setting Also worth knowing when testing
|
|
Tested this at 5120x2160 on an RTX 5090 (driver 621.14, Windows 11), DLAA, Streamline 2.14.16 matching the pinned hashes. Frame generation works: 2x showed ~91 fps from ~46 game fps, 3x 134–152 from 45–51, 4x 146–157 from 37–39. A few things from the run, in order of how much they hurt: Every exit and every Restart crashed with DLSS Frame Gen on: an access violation inside NVIDIA's DLSS-G (read at 0x48). Nothing calls slShutdown, so the process exits with Streamline running, and the DLSS-G plugin tears its frame generation down in its DLL detach, after ExitProcess has already ended the threads it needs. What fixed it here: from mkwvk_close_asked (before the host's ExitProcess), take the present lock (with a ~2 s timeout so a stuck present can't hang the exit), wait for the device to go idle, set DLSS-G's mode to off, then slShutdown, and have every dlssg_* call do nothing after that. Clean exits and restarts afterwards, with "Streamline shut down for the exit (result 0)" in the log. Happy to share the diff. |
…ltrawide DLAA, RX 9070 XT) (#2075) Adds the two lists sent to #2034 so far, each with `nx2/pipeline_states.py add` (only the pipelines no other contributed list holds; the note beside each records who, GPU, build, settings and route). | List | From | Given | New | |---|---|---|---| | `contrib/pmwork-personal-rtx-5090-dlaa-ultrawide` | @pmwork-personal: RTX 5090, Windows 11, 5120x2160 DLAA, DLSS Frame Generation 3x; Kokiri Forest and Castle Town | 1,390 | 506 | | `contrib/yeetbot90-amd-rx-9070-xt` | @yeetbot90: RX 9070 XT, Windows 11, 1440p Lanczos (Auto), frame generation 3x; Kokiri Forest to Goron City | 529 | 137 | The shipped merge goes from 19,038 to **19,571** pipelines. Both lists are version 1 and are converted to version 2 in the merge (none lost). Notes: - pmwork-personal's list was recorded on a fork of main (fc375e9 + #1944 and menu changes) across several sessions with different upscaler settings; the records are the game's draw state, so that only changes which pipelines were met. - yeetbot90's HDR and mods were "unknown"; noted as such. Checked: the zips held only `hookshot_pipeline_states.bin`; `pipeline_states.py check` passes; `test_pipeline_states`, `test_ship_state_list`, `test_pipeline_state_sources` pass. Refs #2034 Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
hckrman101
left a comment
hckrman101
left a comment
There was a problem hiding this comment.
Thanks for this, and for the clear write-up. The Streamline side is put together carefully: manual hooking from load_global, the interposer's signature checked before it loads, Reflex/PCL markers through Streamline, and the renderer's own FG and Reflex standing aside. The two tester reports above (RTX 4080 SUPER, RTX 5090) show it working and pacing well. There are a few things to fix first, though, most of them already found by the testers:
- Every exit and Restart crashes with it on (@pmwork-personal's report). I checked: nothing calls
slShutdown, so DLSS-G tears down in its DLL detach afterExitProcess. The fix they describe (frommkwvk_close_asked: take the present lock with a timeout, wait idle, DLSS-G mode off,slShutdown, then make everydlssg_*do nothing) looks right to me. A crash on every quit is the blocker here. RX_VK_DLSSGstarts Streamline but never turns FG on (Zarigis' bug 2). Confirmed:saved_dlssg()reads it, butopts.dlssgkeeps the ini's value, anddlssg_inputs/dlssg_before_presentboth checkopts.dlssg. Apply it inmenu_initnext toRX_VK_FRAMEGEN_MULT(inline).- DLSS/DLAA fail (
0xbad00004) once Streamline is loaded unlessnvngx_dlss.dllsits besidehookshot.exe(Zarigis' bug 1), so DLSS-G gets no inputs on a stockmake_game.pyfolder. I'd rather find out why NGX ignoresvk_dlss.cpp's path list (probably because Streamline'ssl.commonstarts NGX first) than copy the DLSS runtime beside the exe, because main keeps it inngx\on purpose (place_ngx_runtime: "nothing beside the exe"). Either way the log line shouldn't blame the GPU. tests/test_upscale_independent.pybreaks. It cutsframegen_onout ofvk_framegen.cppand compiles it againsttests/upscale_independent_test.cpp, which has nodlssg_active(). That file needs a stub, the same asframegen_resources_test.cppgot. CI didn't run on this PR ("no checks reported"), so please runpython -I -B tests/run_all.pylocally.motionVectorsInvalidValue = 0.0fmarks every pixel with exactly zero motion as invalid, which is anything still under a still camera. Streamline's default (INVALID_FLOAT) seems to be what's wanted (inline).
Overlap:
- #1831 also adds
gpu/vk/vk_dlssg.cpp/vk_dlssg.h(through NGX, with a different API), so whichever lands second has a hard conflict on the same file names. You already offered this as an alternative. Which route to take is @NotRexed's call, so I'm not approving either one over the other. - The branch is 63 commits behind main. Within this series, #1940/#1941/#1942 touch
vk_temporal.cpp,vk_replay.cpp,vk_framegen.cppandvk_menu.cppnear the same spots, and #1945/#1946 touchvk_menu.cpp/.h(Options). I expect textual conflicts only, but a rebase on main is due anyway.
Licensing / redistribution: this matches how main handles DLSS. scripts/sl_sdk.py mirrors ngx_sdk.py: a pinned release, SHA-256 on the zip and on every file taken out, NVIDIA's notice printed, --accept-nvidia-license or a typed yes required, the licenses saved beside the files, the zip deleted afterwards, sl-sdk/ and the zip gitignored. Nothing from NVIDIA is committed, linked (only wintrust/crypt32) or shipped; the release workflows don't set HOOKSHOT_SL_SDK, and the packaging only picks up the script. FSR 4's "bring your own DLLs" model is stricter still, and this is closer to the NGX one, which is fine. Two small points:
- When the Builder gets a tick for this, it needs the same license text the DLSS tick has in
MainWindow.xaml.cs(and the CONTRIBUTING note should say so, as the NGX section does). --accept-nvidia-licenseinmake_game.pynow stands for both NVIDIA license sets. That's fine since each fetch has its own flag, but the help text could say which licenses it accepts for which fetch.
Nits:
saved_dlssg()hard-codesdlss_frame_gen=, which only matches becauseini_keyderives it from the row's name. Add an explicitini_keyentry foropts.dlssgso renaming the row can't silently break the pre-Vulkan read.- The comment at
pathsToPluginssays no OTA plugins, but pmwork saw the NVIDIA app's DLSS override load OTA plugins from ProgramData anyway. A log line when that happens would help. - Row help: Streamline reports up to 6x on RTX 50 but the row stops at 4x. Also, as both testers saw, the Reflex cap divided by the multiplier caps the game's own rate (about 39 fps at 4x on 165 Hz), and DLAA at 4K can overload even a 4080. Both are worth a sentence in the help.
| /// The row's value as saved, read before the menu reads its file (menu_init comes after the instance): 0 off, 1 to 3 (2x to | ||
| /// 4x). RX_VK_DLSSG=<0..3> over it | ||
| int saved_dlssg() { | ||
| if (const char* e = getenv("RX_VK_DLSSG")) return std::clamp(atoi(e), 0, 3); |
There was a problem hiding this comment.
This makes Streamline start, but opts.dlssg still comes from the ini, so with dlss_frame_gen=0 (or no ini) and RX_VK_DLSSG=1, dlssg_inputs returns early and dlssg_before_present sees "the row is off" every frame, and nothing is logged (Zarigis' bug 2). Applying RX_VK_DLSSG over opts.dlssg in menu_init, where RX_VK_FRAMEGEN_MULT is applied, fixes it and matches the readme.
There was a problem hiding this comment.
Fixed: menu_init now applies RX_VK_DLSSG over opts.dlssg next to RX_VK_FRAMEGEN_MULT, before restart_init, so the env var both starts Streamline and turns frame generation on. test_dlssg.py checks both places.
| logf("vk: Streamline did not start (result %d): DLSS Frame Gen off", (int)r); | ||
| return nullptr; // (the module stays loaded: Streamline may have hooked something already) | ||
| } | ||
| s.active = true, s.why = DLSSG_OK; |
There was a problem hiding this comment.
Nothing ever undoes this: there is no slShutdown anywhere, so the DLSS-G plugin tears down in DLL detach after ExitProcess has ended its threads. That's the access violation on every exit/Restart that pmwork-personal reported. You need a shutdown from mkwvk_close_asked (present lock with a timeout, idle, DLSS-G off, slShutdown, then every dlssg_* is a no-op after it).
There was a problem hiding this comment.
Added dlssg_shutdown(), called at the end of mkwvk_close_asked. It stops new calls, drains an in-flight Reflex sleep without the present lock, takes the present and queue locks with a 2 s limit, waits for idle, turns DLSS-G off, then calls slShutdown, once. Every dlssg_* is a no-op afterwards. slShutdown is now one of the required exports. In the game (RTX 5080, 2x and 3x): six window closes, each result 0, no crash; a Restart with Streamline running is still to try.
| c.cameraFar = f.far_z > c.cameraNear ? f.far_z : 100000.0f; | ||
| c.cameraFOV = f.fov_y > 0 ? f.fov_y : 1.0f; | ||
| c.cameraAspectRatio = p[0] != 0 ? std::fabs(p[5] / p[0]) : 16.0f / 9.0f; | ||
| c.motionVectorsInvalidValue = 0.0f; |
There was a problem hiding this comment.
With 0 as the invalid value, every pixel with exactly zero motion counts as invalid, which is everything static while the camera is still. Unless that was on purpose, leave Streamline's default (INVALID_FLOAT, "no invalid value").
There was a problem hiding this comment.
Not on purpose. Removed, so Streamline's default (INVALID_FLOAT, no invalid value) applies, and the test keeps it unset.
| static const int allow = !getenv("RX_VK_FRAMEGEN") || atoi(getenv("RX_VK_FRAMEGEN")) != 0; | ||
| return framegen_wanted(opts.framegen, allow) && !dev.offscreen && !fg.failed; | ||
| // (hookshot: off while NVIDIA Streamline runs DLSS Frame Generation, vk_dlssg.cpp: it makes the frames then) | ||
| return framegen_wanted(opts.framegen, allow) && !dev.offscreen && !fg.failed && !dlssg_active(); |
There was a problem hiding this comment.
tests/test_upscale_independent.py cuts this function into tests/upscale_independent_test.cpp, which has no dlssg_active(), so that test stops compiling. It needs a bool dlssg_active() { return false; } stub, like the one framegen_resources_test.cpp got.
There was a problem hiding this comment.
Added bool dlssg_active() { return false; } to tests/upscale_independent_test.cpp; test_upscale_independent passes again.
|
Additional RTX 5070 Ti / Windows11 data point:3440x1440, DLSS Quality2294x960, Streamline2.14.16 pinned runtimes. I tested this PR merged locally with mainf8efe057/API14, not the PR head alone. I reproduced the exit access violation twice (read at0x48 inside nvngx_dlssg.dll). Calling slShutdown from mkwvk_close_asked fixed normal exits locally, consistent with @pmwork-personal's report. One concurrency detail: dlssg_sleep drops the state mutex while executing SDK sleep. Shutdown must stop admitting calls and drain that in-flight sleep before slShutdown. Do not hold the present lock while waiting for that sleep to return. My local causal fixture blocks SDK sleep, starts shutdown and verifies that idle/shutdown are delayed until sleep returns; it also checks no trailing marker after shutdown begins and idempotent close. It fails against my first shutdown attempt and passes after this coordination. Use device_wait_idle_locked() so the recorder boundary is preserved. Active-FG normal exits returned0. A stuck-device timeout and Restart remain untested. Matched local snapshot: about60native/120displayed fps before pause and after stabilization. First second after inventory return:59.6native/111.4displayed, displayed P95=12.55ms; inventory about60/60. PresentMon2.6, no simultaneous pixel capture, one run per mode. Presented HUD/inventory text remained legible in inspected frames; this does not identify an individual generated frame or cover the campaign. Off/integrated/Streamline controls used the same host/backend/checkpoint, without a claim of an overall winner or input latency. The runtime-beside-exe, RX_VK_DLSSG option and foreground prerequisites reported above also reproduced. Sharing the shutdown findings here rather than opening an overlapping FG integration. |
8fd3ebc to
c38b896
Compare
|
Thanks for the careful review. All five are addressed, rebased on main d7a7306, still one commit:
Nits:
On #1831: agreed, that's NotRexed's call. |
|
@Zarigis Thanks for the RTX 4080 SUPER numbers and the detailed write-ups. Both bugs are fixed in the update:
The DLAA-at-4K hint is in the row's help now. |
|
@pmwork-personal Thanks for tracking the crash down and for the clear recipe; the update follows it:
5x/6x, a per-exe OTA opt-out, and the actor-motion layouts from #1831/#1834 are left for later. The exit fix held in six window closes here (2x and 3x, |
|
@Arkex-ai Thanks for the RTX 5070 Ti data and the concurrency note, which changed the design. |
…manual hooking, Reflex) DLSS Frame Generation gives RTX 40 and 50 cards smoother motion than the renderer's own interpolation, at less latency, with NVIDIA's network making the frames and Streamline's Reflex pacing them. - gpu/vk/vk_dlssg.cpp/.h: NVIDIA Streamline 2.14 with manual hooking. With the new DLSS Frame Gen row set (read from hookshot_options.ini before Vulkan starts; RX_VK_DLSSG over it), sl.interposer.dll's vkGetInstanceProcAddr takes vulkan-1.dll's place in load_global, after NVIDIA's signature on it is checked. Each frame: the frame chain's depth and motion tagged, constants from the scene camera (no invalid motion value: Streamline's default), PCL markers and Reflex's sleep through Streamline, frame generation on for game frames (off with the Insert menu open, and after 60 frames without inputs: a frame now and then without them keeps it on), and a Reflex limit on the frames shown at refresh - refresh^2/3600 (116 at 120 Hz). The FPS counter counts the made frames and shows the game's own rate. Not started in a --prepare-shaders run (no window, no close to shut it down). - The exit: Streamline is shut down from mkwvk_close_asked (the window's close, Quit and Restart, before the host's ExitProcess). Left to the DLL detach, the DLSS-G plugin tore its frame generation down after ExitProcess had ended its threads: an access violation (read at 0x48 in nvngx_dlssg.dll) at every exit and Restart. dlssg_shutdown lets no dlssg_* call in any more, waits for a Reflex sleep under way with the present lock free, takes the present lock and the queue's (2 s at most each: past that Streamline is left as it is and the exit goes on), waits for the device to idle, turns DLSS-G off and calls slShutdown, once; the locks stay held until the process ends. - DLSS beside it: NGX starts once in a process, with the folders of its first start, and with Streamline that is sl.common's. So Streamline's pathsToPlugins name the folders vk_dlss.cpp gives NGX for DLSS's runtime (RX_VK_DLSS_DIR, the game folder's ngx, %LOCALAPPDATA%\Hookshot\ngx) after the plugins' own; before, DLSS and DLAA failed with 0xbad00004 unless nvngx_dlss.dll was also beside hookshot.exe or NGX had a DLSS of its own downloads. vk_dlss.cpp says that case as NGX's folders, not the GPU. The log says when the NVIDIA app's DLSS override has Streamline use NVIDIA's own plugins from ProgramData. - The inputs come from the temporal upscaler's frame chain (vk_temporal.cpp: DLSS, DLAA, FSR 4), after its engine drew; the chain now keeps the scene camera's projection, and its depth and motion are a transfer's source while Streamline runs. - vk_reflex.cpp, vk_framegen.cpp, vk_fsr3fi.cpp: the renderer's own Reflex and frame generation stand aside while it runs. - Insert menu: the DLSS Frame Gen row (Off, 2x to 4x; restart; saved as dlss_frame_gen by ini_key, which vk_dlssg.cpp reads before Vulkan), shown in a build with Streamline on an NVIDIA GPU; RX_VK_DLSSG turns it on for the run too (menu_init). Its help says why it is not running when it is set, that the game's own rate is held to the shown frames' limit divided by the multiple, and that DLAA at a 4K output can be too much. - nx2/build.py: HOOKSHOT_SL_SDK=<folder> builds vk_dlssg.cpp with Streamline's headers (Windows only) and links wintrust/crypt32 for the signature check. nx2/make_game.py copies the runtime (sl.interposer, sl.common, sl.dlss_g, sl.reflex, sl.pcl, nvngx_dlssg, NvLowLatencyVk) beside hookshot.exe, and --fetch-sl-sdk fetches the SDK first; --accept-nvidia-license's help names the licenses of each fetch. - scripts/sl_sdk.py: Streamline's binaries are only in its release zip, so it downloads the pinned v2.14.16 zip after NVIDIA's licenses are accepted, checks it and every file it takes out against their SHA-256s, and deletes it. NVIDIA's files are never committed or shipped. - Tests: tests/test_dlssg.py (the frame limit, the row and its key, RX_VK_DLSSG, the stubs, the hand-over points, NGX's folders, and the exit run with a stand-in SDK whose Reflex sleep blocks: the exit waits for it with the presents free, lets nothing in, then idle, DLSS-G off, slShutdown, once; a stuck present or sleep gives up after the wait), tests/test_sl_sdk.py, tests/test_make_game_sl.py; test_reflex_swapchain_lifetime, test_shader_stamp_cache, the framegen and upscale_independent fixtures, test_audio_output_menu and test_stick_dead_zone know vk_dlssg.cpp and the row. Not done: a Hookshot Builder tick for it (the command line only); rows past 4x (Streamline offers up to 6x on RTX 50). Tested: RTX 5080, driver with HAGS on, 120 Hz G-SYNC, NVIDIA app VSync on: 116 fps shown, 58 of the game's, no tearing, on v2.0.13 (where the inputs came from vk_dlss.cpp). Streamline does not support VSync with frame generation on Vulkan, but the driver's works; Frame Warp is not available on Vulkan. Testers' runs of the first version (RTX 4080 SUPER, RTX 5090, RTX 5070 Ti) found the exit crash, the 0xbad00004 and RX_VK_DLSSG. On main d7a7306: compile-checked with and without Streamline (and vk_dlss.cpp with and without NGX), and the game-free tests; the exit shutdown and the NGX folders are not yet tried in the game. Assisted-by: Claude Code Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
c38b896 to
e2710eb
Compare
|
Rebased onto current main. One conflict: main now colours the FPS counter by the game's own rate against its cap when frame generation is on; that rule now covers DLSS Frame Generation too ( |
|
Thank you for your contribution, I will close this for now as we are going with NGX for framegen at the moment, since we already have NGX and it supports Linux. Please test those PRs instead! |
… up (#2118) On an RTX 5090 at 5120x2160 the built-in frame generation couldn't follow motion. Its block search covers ±32 px per game frame, so a camera pan's vectors fell to zero and every made frame was a cross-fade of two game frames: smoother counters, same stepping motion. This PR makes those frames with NVIDIA's DLSS Frame Generation through NGX, from the temporal upscaler's own depth and motion. It also fixes two pacing problems that made 3x feel uneven even once the frames were right. **What changed (five commits, each on its own)** 1. **`fix(vk)`: present 3x and 4x sets ahead too.** #1702's present-ahead (a set's copies go to the GPU together, and only the presents wait for their times) ran for 2x only. At 3x and 4x each made frame's copy still queued behind the game's next frame and reached the screen unevenly. It now runs for any paced set the swapchain can hold, and at 3x/4x the swapchain asks for one spare image per frame (5 and 6 images on NVIDIA). 2. **`feat(dlss)`: DLSS-G through NGX** (`vk_dlss.cpp`, beside DLSS on the same NGX session, no Streamline): - **Inputs.** The chain's depth and motion (actor motion included) are copied at its evaluation, with the camera. The scene's projection is now kept beside its view-projection (camera block 0x70). - **Output.** At the present, DLSS-G writes the frame generation row's made frames into the renderer's own set, which the display thread shows and paces as before. - **Fallback.** A set it doesn't make is made by the existing methods, and the log says every 600 presents how many sets DLSS-G made and why it didn't make the rest. - **Motion units.** The motion goes to NGX **in render pixels** (MvecScale 1), the same as to DLSS. The header's "[-1, 1]" is Streamline's convention. With 1/render size, DLSS-G saw no motion and made cross-fades again; I captured frames to confirm, and it was 0.2/255 off a pure blend at t = 1/3 and 2/3. - **Exit.** The feature is let go with the queue idle. - **Runtime.** `nvngx_dlssg.dll` is looked for where DLSS's runtime is. - **Menu.** A **Frame generation by** row: Auto (DLSS-G where it runs, otherwise the built-in one) or Built-in. Both frame generation rows now sit under the Upscaler, as asked in #dev-work. The row changes at once, with no restart. `RX_VK_DLSS_FG=0/1` overrides it for a run. 3. **`fix(vk)`: V-Sync Auto presents FIFO with DLSS Frame Generation at 2x and up.** Auto was mailbox, which on NVIDIA is Fast Sync: below the refresh rate (~135 shown on 165 Hz) frames land on the refresh grid, so evenly timed presents still judder. With DLSS-G making the sets, Auto now asks for FIFO, which a VRR display shows as presented. The built-in frame generation and other GPUs keep Auto as before (from review, item 5). On and Off are unchanged. The swapchain is rebuilt when this changes, as it already is for V-Sync. 4. **`feat(vk)`: NVIDIA Reflex with frame generation** (added after review). Reflex was off whenever frame generation was on, so its held-back frame added its whole delay (nullifyshot measured +12 to +18 ms at 4x on an RTX 5080). The game's frame calls stay on: the sleep, the simulation markers and render submission. Each set keeps the game frame it was made at, and only that frame is presented with its present id and present markers; made frames get neither. Present ids only go up. Without frame generation nothing changes. On the 5090 at 3x, Reflex's own timings now appear under DLSS-G: simulation start to present end ~41 ms, against ~16 ms with frame generation off (details in the comments). 5. **`fix(dlss)`: DLSS-G only where it runs** (from review). NGX isn't asked for frame generation's device extensions, or started at a present, on a device without NVIDIA's extensions. V-Sync Auto's FIFO is for DLSS-G only. Frame generation at Off or Auto lets DLSS-G's feature go. A barrier follows a refused frame before the built-in passes write the same images. **How it was tested** - RTX 5090, driver 621.14, Windows 11, 5120x2160 @ 165 Hz with G-SYNC, DLAA, frame generation 3x, Kokiri Forest: - DLSS-G made every set in gameplay (600/600 per log window), ~135 fps shown from ~45 game fps. - Frames 7–9 ms apart on the display thread. The pacing felt as even as #1944's Streamline path on the same machine. - Clean exits. - `tests/run_all.py` (Windows, at 90c4683): 340 passed, 28 skipped, 3 failed. The same 3 also fail on untouched main at this branch's base here: - `test_recorder_sync`'s exit case; - `test_precompile_compute_guard`; - `test_finish_output` (a `NameError` in `make_game.py`; it passes on current main). - Updated tests: - `test_present_ahead.py`: 3x/4x with and without enough spare images. - `test_vsync_row.py`: the present-mode choice with DLSS-G and with the built-in frame generation. - `test_upscaler_rows.py`: the new row order. - `present_source_crop_test.cpp` and `test_present_semaphores.py`: stubs for the new calls. - `test_recorder_sync.py`: knows NGX's DLSS-G create/evaluate calls as direct recordings with their B_DIRECT boundary. **Not tested / known** - Only one GPU (RTX 5090), Windows only. - Not checked on a fixed-refresh (non-VRR) display, where FIFO would hold frames to the refresh. - **Follow-up:** Reflex's "to the GPU's end" reads 243–442 ms under DLSS-G (nullifyshot's measurement). Tagging GPU submits with `VkLatencySubmissionPresentIdNV` should fix it; I'll do that as its own PR. - The HUD is drawn into the same picture (no HUD-less input), as with FSR 3.1's interpolation. I didn't see HUD smear at 3x, but I didn't look for it specifically. - Link's limbs still step a little when jumping; that's the actor motion, not frame generation. #1831/#1834's single/pair/rigid actor-motion layouts fix his shield and hair here, so I'd suggest those as their own PR. I'll follow up on the arms and feet. - Relation to the open frame generation PRs: - **#1944 (Streamline):** I tested it on the same machine, and its exit crash and fix are in my comment there. - **#1831 (NGX):** it was the starting point for this path; its MvecScale 1 was right. - **Overlap:** happy to fold this into either if the maintainers prefer. 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: pmwork-personal <315945841+pmwork-personal@users.noreply.github.com> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
…ters' motion (#2305) AMD FSR 3.1 makes its own inputs (`vk_fsr3_inputs.cpp`), and the only motion it gets is the camera's. So anything that moves differently from the camera is given the motion of what's behind it: Link while he walks, his swinging arm, his shield, and the people around him. DLSS and FSR 4 already get those characters' own motion on `vk_temporal.cpp`'s chain. This PR puts FSR 3.1 on that chain too, as a third engine, so all three temporal upscalers share the same inputs. **What changed** - `vk_fsr3.cpp/.h`: `kChainEngine`, FSR 3.1 as a `TemporalEngine` (slot 2). - It takes the chain's colour, depth, motion (with the actor pass drawn over it), jitter and jitter phase count. - It builds FSR's depth constants from the unjittered projection. - Its picture is copied into the chain's output and squared back by `dlss_out.comp` as before. - `fsr3_chosen()` is "the row or `RX_VK_FSR=1` picks FSR 3.1". `fsr3_wanted()` now means "FSR 3.1 on its own inputs", which is false while it's on the chain. - `fsr3::prepare_sizes` and `fsr3::release` let the chain size and free FSR's images. - `vk_temporal.cpp/.h`: - `temporal_mode` returns `FSR3` for the row's FSR 3.1 before it falls back to the game's own; `engine_of` gets a third slot. - Two new `TemporalEngine` fields: - `fsr_depth`: the chain reverses depth for FSR's passes (built with `INVERTED_DEPTH`) where the projection puts the near plane at 0. - `retry`: a failed evaluation hands that frame to the network instead of turning the engine off for good. FSR 3.1's constants come from the frame's stream, which can run out for one frame. - `TemporalFrame` carries the unjittered projection and the jitter phase count. - The chain's output image gets `TRANSFER_DST`. - `RX_VK_FSR_JITTER=0` still turns FSR 3.1's jitter off on the chain. - `shaders/dlss_prep.comp`: - Flag bit 3 reverses depth (`1 - d`). - A NaN in the scene colour is written as black, following `fsr3_prep.comp`'s rule, because one NaN spreads through an upscaler's history. - The header was regenerated with `glslangValidator -V --target-env vulkan1.3 --vn DLSS_PREP_COMP_SPV`, the command `test_dlss_actor.py` checks. - `RX_VK_FSR3_CHAIN=0` gives FSR 3.1 its own inputs, as before, for comparisons (`docs/development.md`). The readme's FSR 3.1 section describes the new inputs, and says what still trails: things that aren't skinned, such as Navi's body. - Tests: - `temporal_jitter_test.cpp` covers FSR 3.1's slot and its failure gate, and a new run (`fsr-jitter-off`) where only `RX_VK_FSR_JITTER=0` is set: FSR 3.1 on the chain gets no jitter, and DLSS and FSR 4 keep theirs. - `test_upscaler_rows.py` checks that the row's FSR 3.1 runs on the chain before the fallback to the game's own, that `fsr3_wanted` is off while it does, and that `engine_of` has its slot. - Both fail on main. A mutated jitter gate fails `fsr-jitter-off`. **Testing** - `python -I -B tests/run_all.py`: 363 passed, 0 failed, 11 skipped (Linux; the skips are Windows-only, need a Vulkan 1.3 device, or need dotnet). - In game: RX 9070 XT 16 GB, Mesa RADV 26.2.6, Gentoo Linux (kernel 7.3.3), KDE Plasma Wayland, windowed, 2560x1440 output, 60 fps cap. - Host v2.1.0-main-local (source fc2ed0d0). Renderer: main 49f75dc plus this branch. - "Own" means the same build with `RX_VK_FSR3_CHAIN=0`, which is what main does. - Every chain run logged `vk: fsr3 on the chain: 600 frames drawn ... depth reversed for FSR by the chain`, with no evaluation failures. | FSR 3.1 | Kokiri Forest, 28 s camera turn (4 runs each) | Castle Town market at noon (3 runs each) | |---|---|---| | Own inputs (as main) | 59.2 fps, 1% low 46.4 | 53.1 fps | | On the chain | 59.3 fps, 1% low 48.1 | 52.5 fps | Market runs vary by about 0.9 fps between runs, so the chain costs nothing measurable. - Picture: consecutive 2560x1440 shots at Kokiri's lost-woods path, standing and walking, compared on the same build. Edge flicker is the mean frame-to-frame change of edge pixels, 0-255, while standing. | FSR 3.1 | Edge flicker, standing | 1-px outlines per MP | Halos per MP | |---|---|---|---| | 100%, own | 2.17 | 32 | 45 | | 100%, chain | 2.20 | 29 | 50 | | Quality (67%), own | 3.01 | 12 | 25 | | Quality (67%), chain | 1.67 | 2 | 9 | Four independent reviewers of the walking sheets saw no visible difference at 100%. At 67%, they found the chain slightly better: smoother edges on Link's shield and tunic hem, and slightly softer shield texture. No new artefacts. The clearest evidence is the motion field itself (below). On main, Link isn't in it at all. **Not tested:** - other GPUs and drivers; - Windows; - Steam Deck, where #2110 would make FSR 3.1 the default; - frame interpolation (`RX_VK_FSR_FG=1`), which keeps using `vk_fsr3_inputs.cpp`; - switching upscalers repeatedly while playing (release and remake are code-reviewed, not exercised in game); - HDR: the chain still clamps FSR 3.1's colour at the paper white, as main does (see below). **Screenshots** (from the game's output image before the present path, 2560x1440) The motion vectors the upscaler gets while Link walks (`RX_VK_DLSS_DEBUG=motion`). Left: camera motion only (`RX_VK_DLSS_ACTORS=0`, which is what FSR 3.1 got on main). Right: on the chain, with Link's body, arm and shield carrying their own motion:  Link walking: own inputs vs the chain, at 100% and at 67% (2x zoom crops):  **Merge notes** (from test merges against every open PR that touches these files; textual conflicts show up in git, semantic ones don't) - **Depth is reversed for FSR 3.1 on the chain.** `TemporalFrame::depth_reversed` says so. Anything that reads the chain's depth must check it rather than assume plain depth: - #2118 (DLSS frame generation through NGX, now on main): done in this PR. Commit ca2e302 passes `depth_reversed` to NGX's `depthInverted`, and `test_upscaler_rows` checks it. - #1944 (DLSS frame generation through Streamline) needs the same: `depthInverted` from `depth_reversed`, not a hard-coded false. - #1831/#1833: `nr_pre` passes `s.depth` on, and the NGX backend's `DepthInverted` should follow it too. - **#2145** adds `bool native_aa` to `TemporalEngine::prepare`. `chain_prepare` in `vk_fsr3.cpp` needs the same parameter, or the build breaks. git merges the two cleanly. FSR 3.1 never has native AA. - **#1653** (reactive mask) is on main now and merged into this branch. Commit 7ceae45 makes the mask in `chain_evaluate` the same way DLSS does on the chain, and `test_reactive_mask.py` checks it. Checked in game with `RX_VK_REACTIVE=1`: the mask is made, and FSR 3.1 draws all 600 frames on the chain. - **#2243** (DLSS HDR, linear light) is on main now (in #2342's squash, a0dfd28) and is merged into this branch. - `dlss_prep.comp` scrubs NaN first, then applies #2243's `max`/`kSquare`/`min`, and keeps the `flags & 8` depth flip. The header was regenerated; glslang 16.4.0 gives the same SPIR-V words as CI's 16.6.0 for `dlss_prep` and `dlss_out`. - `TemporalEngine` is ordered `memory`, `hdr`, `fsr_depth`, `retry`: DLSS's and FSR 4's engines are positional, and `kChainEngine` is designated. - FSR 3.1 doesn't report `hdr` yet, so it keeps the LDR pair. Giving it HDR colour on #2243's mechanism is a follow-up. - **#1831**: - It reads `fsr3_wanted()` to ask whether FSR 3.1 is chosen. After this PR that question is `fsr3_chosen()`. Its `fsr4 = !fsr3_wanted() && fsr4_wanted()` guard and its jitter guard should use `fsr3_chosen()`. The merge is textually clean, so git won't flag it. - In `vk_temporal.cpp`, the output image's usage should become `TRANSFER_DST | TRANSFER_SRC`, and its `proj[16]` and this PR's `projection[16]` should become one field. - **#2166** only matters with `RX_VK_FSR3_CHAIN=0`. - `docs/tested-configurations.md` and `docs/development.md`: rows next to other PRs' rows; keep both. 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
On RTX 40/50 cards, DLSS Frame Generation gives smoother motion than the renderer's own interpolation, at lower latency. NVIDIA's network makes the frames and Streamline's Reflex paces them. This overlaps #1831, which does frame generation through NGX directly and keeps a Streamline path switched off. This is offered as an alternative: happy to close it if you prefer the NGX route.
Since the review (rebased on main d7a7306, still one commit)
mkwvk_close_asked, which the window's close, Quit and Restart all go through before the host'sExitProcess.dlssg_shutdown:dlssg_*calls in;dlssg_sleepdrops the state lock during it) without holding the present lock;device_wait_idle_locked), turns DLSS-G off, then callsslShutdown.dlssg_*call does nothing afterwards, and both locks stay held until the process ends, so nothing reaches Streamline's hooks after it shut down.--prepare-shadersrun (no window, so no close) no longer starts Streamline.0xbad00004withnvngx_dlss.dllonly inngx\: NGX starts once per process and keeps the folder list from that first start. With Streamline, the first start issl.common's when the device is made.sl.commonpasses NGX the plugins' folder plus every entry ofPreferences::pathsToPlugins(commonEntry.cppin the 2.14.16 source). SopathsToPluginsnow lists, afterbin\(the first folder with plugins still wins for the plugins), the same foldersvk_dlss.cppgives NGX:RX_VK_DLSS_DIR, the game folder'sngx,%LOCALAPPDATA%\Hookshot\ngx. Nothing is copied beside the exe. With Streamline running,vk_dlss.cppnow logs this case as NGX not finding the runtime in its folders instead of blaming the GPU. The game folder is also taken wide now, not through the ANSI code page.RX_VK_DLSSGis applied overopts.dlssginmenu_init, next toRX_VK_FRAMEGEN_MULT, so it also turns frame generation on for that run.motionVectorsInvalidValueis left at Streamline's default (INVALID_FLOAT).tests/upscale_independent_test.cppgets thedlssg_active()stub.ini_keyhas an explicitdlss_frame_genentry, whichsaved_dlssgreads.--accept-nvidia-license's help names the licenses for each fetch.What changed
gpu/vk/vk_dlssg.cpp/.h: Streamline 2.14 with manual hooking. With the DLSS Frame Gen row set,sl.interposer.dll'svkGetInstanceProcAddrreplacesvulkan-1.dllinload_global, after NVIDIA's signature is checked. The row is read fromhookshot_options.inibefore Vulkan starts;RX_VK_DLSSGoverrides it.vk_temporal.cpp: DLSS, DLAA, FSR 4) after its engine drew. The chain keeps the scene camera's projection. Main's present-ahead split is respected: the hook sits right beforepresent_out.vk_reflex.cpp,vk_framegen.cpp,vk_fsr3fi.cpp: the renderer's own Reflex and frame generation stand aside while it runs.vk_main.cpp:mkwvk_close_askedends withdlssg_shutdown().vk_dlss.cpp: the0xbad00004log line with Streamline running.nx2/build.py:HOOKSHOT_SL_SDK(Windows) buildsvk_dlssg.cppwith Streamline's headers and links wintrust/crypt32.nx2/make_game.py: copies the seven runtime DLLs besidehookshot.exe;--fetch-sl-sdkfetches the SDK first. The DLSS runtime stays inngx\.scripts/sl_sdk.py: downloads the pinned v2.14.16 release zip after NVIDIA's licenses are accepted, and checks the zip and every extracted file by SHA-256. NVIDIA's files are never committed or shipped.test_dlssg.py,test_sl_sdk.py,test_make_game_sl.py.test_dlssg.pynow also covers the exit. It cutsState,dlssg_marker,dlssg_sleepanddlssg_shutdownout ofvk_dlssg.cppand runs them against a stand-in SDK whose Reflex sleep blocks:slShutdown;Each of these fails against a mutant that drops the matching step. It also checks
RX_VK_DLSSGinmenu_init, the NGX folders, and the close and Restart path inhost/display.c.test_reflex_swapchain_lifetime,test_shader_stamp_cache, the framegen and upscale_independent fixtures,test_audio_output_menuandtest_stick_dead_zoneknowvk_dlssg.cppand the row.Testing
vk_dlssg.cppcompiles with and without Streamline,vk_dlss.cppwith and without NGX, and the other touched files compile.tests/run_all.py dlssg sl_sdk make_game_sl make_game_ngx ngx_sdk upscale_independent framegen present_ reflex fps_rate menu stick_dead_zoneon Windows: 32 passed, 3 failed.test_present_aspect_vulkanfails the same way on untouched main.test_reflex_swapchain_lifetimeandtest_present_semaphoresdefault to clang++, which this PC lacks. Both pass with--cxx g++.Streamline shut down for the exit (result 0)and exited, and Windows recorded no crash. A Restart from the menu relaunched fine from a session without Streamline; a Restart with Streamline running was not tried. DLSS/DLAA withnvngx_dlss.dllonly inngx\still needs a PC without NVIDIA's own DLSS download: this one has the NVIDIA app's override, which is likely why DLSS worked here without the copy.Not done / not tested
vkDeviceWaitIdlehas no timeout.sl_sdk.pywas checked against a local copy of the zip, not fetched from CI.Screenshots
HDR10 captures, tone-mapped to SDR for this page (paper white to white, highlights rolled off up to 800 nits).
🤖 Generated with Claude Code