Repository navigation
feat(vk): neural rendering at the present, through NGX direct and OpenDLSS-NR, with a Neural tab - #2820
Open
notquiteog wants to merge 6 commits into
Open
feat(vk): neural rendering at the present, through NGX direct and OpenDLSS-NR, with a Neural tab#2820notquiteog wants to merge 6 commits into
notquiteog wants to merge 6 commits into
Conversation
OpenDLSS-NR (https://github.com/maanHimself/OpenDLSS-NR, MIT, Copyright 2026 maan) is a Vulkan implementation of the DLSS 5 neural rendering network. The Neural tab's Open backend, in the commits after this one, compiles its host sources into the renderer and runs its kernels, so its files are here first, unmodified, at commit 9d08f418: - src/: the compute context, the model directory loader, the kernel wrappers, the 71-block graph and the CPU reference of its arithmetic; and upstream's dlss5vk tool, which only the correctness harness links; - shaders/: the GLSL kernels; - scripts/ptx/: the Python generators of the PTX kernels; - LICENSE, NOTICE and PROVENANCE.md, which says what was left out and how the files are built here. No model weights are here and nothing here produces them: the network reads a model directory the player supplies. Nothing is taken from any NVIDIA binary. Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
…enDLSS-NR DLSS 5 neural rendering is a model run over each finished frame: colour in, colour out, not an upscaler and not ray reconstruction, so it works on a rasterized game. This adds the two backends that run it and the one call the present makes; the menu's tab is the next commit, so here it is off unless RX_VK_DLSS_NR=1 is set. Where it runs: do_present (vk_replay.cpp), once for each of the game's frames, on the picture's part of the window image, before frame generation and before the menu is drawn, in the recording after the present's own so its GPU time does not hold the game's next frame back. It returns an image of its own; a frame it cannot do goes on as before. It is left out while the picture is the game's own HDR10. The backends, behind nr:: (vk_dlssnr_ngx.cpp): - NGX direct (vk_dlssnr_ngx.cpp, vk_ngx.cpp): NVIDIA's model as NGX feature 18, over the NGX session vk_dlss.cpp already owns (dlss_ngx_session, dlss_ngx_runtime, dlss_ngx_data_path). Only in a build with the NGX SDK; the stubs link without it. The parameter names are openNR's (MIT), written out as facts. A feature a pass (nr_plan.h). - Open (vk_opennr.cpp, gpu/vk/opennr): OpenDLSS-NR on the renderer's own device, from a model folder the player supplies (nr-model in the game folder): no weights are included and nothing here gets them. Upstream's kernels on NVIDIA (VK_NV_cooperative_matrix2, with the PTX route where VK_NV_cuda_kernel_launch is), and the khr route (gpu/vk/opennr/khr) on VK_KHR_cooperative_matrix alone. Its temporal path reads the scene's motion (fsr3_inputs::latest). Both are given the picture alone (vk_nr_scale.cpp), smaller under the Resolution scale, with the model's detail put back at full size (shaders/nr_detail.comp). What is asked of the device: nothing, unless the player asked for neural rendering by the start. NGX is asked for feature 18's device extensions only with the row saved On on an NVIDIA GPU; the open network's extensions and feature structs are requested only with the row saved On and Backend Open (or Auto on a GPU that is not NVIDIA's), a model folder in place, or RX_VK_DLSS_NR_BACKEND=open. With it off, vkCreateDevice is given what it is given today on every vendor. A row turned on later says when a restart is needed. Build: nx2/build.py compiles OpenDLSS-NR's host sources from a patched copy in the build folder (third_party stays upstream's), generates its PTX kernels into opennr_ptx.h, and gives the NGX files NVIDIA's headers; ngx_link adds wintrust (Windows' own) for the refusal report. The SPIR-V headers are scripts/opennr_kernels.py's, checked against their sources by tests/test_opennr_kernels.py with the pinned glslang. The AppImage carries the sources a player's build compiles and OpenDLSS-NR's licence. Checked on Linux, RTX 5060 Ti: the back end builds with and without the NGX SDK; scripts/opennr_harness.py runs every route of the network bit for bit against the GLSL reference on a synthetic model (tests/test_opennr_harness.py, test_opennr_khr.py). Not run: NVIDIA's model (NVIDIA ships no Linux libnvidia-ngx-dlssnr.so), anything on Windows, the khr route on an AMD GPU, and real weights. Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
… settings Neural rendering could only be switched on with RX_VK_DLSS_NR. The Insert menu gets a sixth tab, Neural, between Controls and Debug, with the switch and the model's own settings: - Neural rendering: Off by default. Its hint is the running backend's reason when it cannot run, and says when a row turned on while the game runs needs a restart (the device is made for a backend only when it was asked for by the start). With the row off nothing is asked of NGX or the network. - Backend: Auto, NGX direct, Open (OpenDLSS-NR), saved as nr_backend 0, 1, 2; RX_VK_DLSS_NR_BACKEND=auto|ngx|open over it for a run. Auto's hint says which one runs. - Model, Style, Preset, Passes, Resolution scale, Intensity, Local tone, Local structure, Global tone, Skin structure, Auto mask. Each takes effect on the next frame; the help says which ones are NGX direct's alone. Every row but the switch is saved under an nr_ key (nr_model, nr_style, ...): "model", "preset" or "backend" alone would say nothing in hookshot_options.ini beside quality_preset and dlss_model. None is a Quality preset's row. The control bridge names the tab "neural". The texts are English: the 13 translation tables are not extended here. Checks: tests/test_opennr_backend.py cuts opennr::selected() out of the source and runs it (off with no model folder is "no" on AMD, Intel and NVIDIA, so the device is made as without the backend), and pins the device hooks' gates, NGX's, and the hint's restart line; tests/test_nr_settings_live.py compiles nr_settings() and checks the rows' ranges, the defaults, the Backend numbering and that nothing caches a setting; tests/test_nr_plan.py the Passes and Resolution scale rows. The two fixtures that compile ini_key() get the fields it now names. Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
…s entry docs/neural-rendering.md says what DLSS 5 neural rendering is and is not, where it runs in the frame, what is asked of the device and when (nothing with the row off), NGX direct's parameter set and where it comes from, the open backend (the model folder the player supplies, the frame's passes, the khr route), the Passes and Resolution scale rows, and plainly what was measured and what was not: NVIDIA's model has not produced a frame here, and the open network was measured on synthetic weights. The readme gets the sixth tab, the Neural rows in the options table, a section on the two backends and what each needs, and the credits for OpenDLSS-NR and openNR. CONTRIBUTING.md and docs/development.md name OpenDLSS-NR among the third-party parts. Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
notquiteog
marked this pull request as ready for review
November 12, 2026 18:10
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Hookshot has no neural rendering in the tree: #1590 and #1833 are both open for it, and #1833 (which carries an earlier copy of this code from my fork) cannot be merged because it only builds together with two other PRs. This is the same work as one branch that builds and passes the checks alone on main, with the review points raised on #1833 and #1335 dealt with.
Two things to know before reviewing. NVIDIA ships no Linux
libnvidia-ngx-dlssnr.so, so the NGX direct backend has not evaluated a frame on the PC this was prepared on: it needs a run on Windows with an RTX card before it merges (testing wanted). And it overlaps #1590, which is in review for the same NGX feature: I would rather the maintainers say how they want the two to relate (this as a follow-up for the Open backend and the Resolution scale, or parts of it folded into #1590) than have two PRs compete.What it is
A model run over each of the game's finished frames at the present, before frame generation, with a Neural tab in the Insert menu (On/Off, Backend, Model, Style, Preset, Passes, Resolution scale, Intensity and the tone and structure strengths, Auto mask), every row read live and saved as
nr_<name>.vk_dlssnr_ngx.cpp,vk_ngx.cpp): NGX feature 18 on DLSS's own NGX session. Nothing of NVIDIA's is in the repository; the player's ownnvngx_dlssnr.dllis looked for where DLSS's runtime is.vk_opennr.cpp,gpu/vk/opennr/,third_party/opendlss-nr, MIT): OpenDLSS-NR, the network run by compute shaders, from a model folder the player supplies. Hookshot ships no weights and fetches none.vk_nr_scale.cpp,nr_plan.h): the model on a smaller copy of the picture with its detail put back at full size, which is what makes it affordable (numbers below).Review points from #1833, and what this does about them
vk_camera_motion.h, noscene_frame_inputs. The temporal path takes the scene's motion fromvk_fsr3_inputsas FSR 3.1's interpolation does.VK_EXT_shader_float8asked for with NR off on non-NVIDIA devices." The Open backend's extensions and features are requested only when the player has asked for it by the time the device is made: Neural rendering saved On with Backend at Open (or Auto on a GPU that is not NVIDIA's), a model folder already in place, orRX_VK_DLSS_NR_BACKEND=open. With the row off and no model folder,vkCreateDevicegets exactly what it gets today on every vendor;tests/test_opennr_backend.pypins that. A row turned on later without them says a restart is needed.opennr_*_spv.h. They are compiled by glslang from their sources byscripts/opennr_kernels.py, andtests/test_opennr_kernels.pyrecompiles and compares. Size: of this PR's lines, the vendoredthird_party/opendlss-nrand the generated SPIR-V headers are the large majority; the hand-written part is about 7,400 lines across 56 files, tests and docs included.How it was tested
tests/run_all.py(Linux, main at 65a4d87 merged in): 504 passed, 2 failed, 22 skipped. Two checks fail on this branch exactly as they fail on main at 65a4d87 without it:test_readme_menu_rows(the Skip opening row has no readme line; docs(readme): the option table has the Skip opening row, so main's readme check passes #2857 is open for it) andtest_program_cache_one_variant(its fixture lacks the zstd cache's names since perf(vk): compress the program cache with zstd (RX_VK_PROGRAM_CACHE_ZSTD) #1630).test_readme_menu_rowsnow reads the Neural tab too, and the readme's option table has a line for each of its rows.scripts/opennr_harness.py model, random weights of the right shapes: it proves the path, not the picture): RTX 5060 Ti 16 GB, driver 615.71.09, Linux (COSMIC, Wayland), Kokiri Forest, 2048x1152 output, DLSS, V-Sync Off.Scale changed in play both ways, the row off and on again, Backend to NGX direct (refused with the reason below, the game's picture shown) and back to Auto (Open again), DLSS Frame Generation at 2x on top (600 of 600 presents made by DLSS-G): no device lost, no error.
NGX answers DLSSNR.Available = 0 ... NVIDIA has not shipped it for Linux yet.Not tested
nvngx_dlssnr.dll). Compiled and read only.khrroute), Intel, and Windows at all.Known and left
🤖 Generated with Claude Code