Repository navigation
fix(shaders): noperspective varyings interpolated right on AMD (Royal Family Tomb water); sirit names an undefined operand - #1830
Conversation
ghost
left a comment
There was a problem hiding this comment.
Thanks for splitting the super build up. Several of these fixes are real, but this branch doesn't stand on its own yet, and about half of the lowering changes overlap with open or merged PRs in the same file.
Blocking
- Build break:
upload_unswizzle_gpucallsmemory_guest_contained(), but nothing on this branch declares or defines it. It's declared invk_internal.honsuper-build/all, which isn't part of this PR (see inline). - Three registered tests fail as written.
run_all.pypicks up everytests/test_*.py. These are probably the "tests that need a compiler" that didn't run on your machine:test_lower_suld_1d.pyexpects a 1D coordinate in R254 to lower, and 1D_BUFFER / 1D_ARRAY to be refused. Both checks now go the other way.test_lower_lea_hi_constant.pyexpects a high source (Rc not RZ) and a zero shift to be refused. The new LEA.HI block accepts both.test_shader_stamp_cache.pynow asserts thatnx2/build.pyhasstreamline_sdk_pathandopennr_jobs. Neither is on this branch; they come from #1833/#1834.
- No CI has run on this branch. It's 26 commits behind main.
Overlap with other PRs (please rebase and drop the duplicates):
- R2UR span: #1806 is merged (9edb3a5) with the same
FieldSpanline, so drop this copy. Main has it after theextended_renamingcheck, and yours comes before it, so git may keep both. - Immediate at bit 32 + compute local memory: #1800 (approved) fixes both. Its bit-32 skip is narrowed to form 4 of the ALU families, and it sanity-checks +0x68 against +0x50 (≤64 KB). The
!is_register[f]skip here is the broad version. It drops the bit-32 registers of TLD4 0xb63/0xf63, TMML 0xb69/0xf69, TEX 0xf60, SHFL.IDX 0x589 and STL 0x987, none of which is in theregister_fieldlist. Their Rb then stops getting its REGMAP prefix whenever another field is high. That's a silent miscompile of programs that lower fine today. Please keep #1800's version and drop these hunks. - LDG/STG bit 78: #1778 widens the same masks and brackets the access with
MEMBAR.GLwhen bit 78 is set. Bits 77-79 all set is.STRONG.GPU, as on the RED/ATOMG cases in this file. Here the qualifier is just dropped. The fences need to stay, so please take #1778's (approved) version. The LDG mask here also lets bit 78 through without bit 77. - Closing self-branch: #1810 changes the same line. Your version (no pending forward branch or call target) is more general than #1810's "previous instruction is an unconditional BRA/EXIT", and I think it's sound: an unconditional self-branch never falls through, and
finishre-checks the relocations. Only one of them should land, though. Two small differences from the EXIT path: it doesn't checkbarrier_tokens, and it allows a call target ≤ index beforemain_exited, which the EXIT path refuses. It also has no test.
Lowering details
- SULD: main already lowers SULD.P.1D, which covers those 17 programs. This hunk adds 1D_BUFFER and 1D_ARRAY, which main's comment says OoT never uses, and there's no test for them. It also gets the coordinate limits wrong. 1D and 1D_BUFFER need one register, so R254 should stay allowed (this is the regression the test catches). 1D_ARRAY needs two (x, layer), so its limit is 253, not 252.
- LEA.HI constant base: the Rc/.X decoding mirrors the 0x211 register form, so the new paths look right. But main's mask also required bits 84-86 and 91-95 to be zero, and this check no longer looks at them, so unknown qualifiers there now pass silently. Please keep the mask, widened only by bit 74 and Rc. Also update the test for what is now accepted. Note that main's comment already says the 40 no-carry programs lower.
- ISBERD.O: the lowering and frontend look fine. Nit:
IsbeReadOutputignores.32(bit 44), so a U8 read would return the whole word.PayloadWordalready clamps, so the extraClampedin the task branch is redundant.
Not in the description (worth their own PR, or at least a mention):
- The noperspective-varying rewrite (
RuntimeInfo::noperspective_outputs, the vertex outputs scaled by Position.w, fragment inputs multiplied by FragCoord.w). The math checks out, but it changes every VS+FS pipeline with a noperspective input on every vendor, not only AMD. upload_unswizzle_gpuno longer imports a pool by default, and the multithreaded CPU unswizzle threshold drops from 16 MB to 4 MB. Both are behaviour and performance changes.texture_touch_misses()has no caller on this branch.
On "70 programs": per the description, that number comes from the in-game programs (0 failed) log on the combined super-build/all branch, from one Kokiri Forest run on NVIDIA. It wasn't measured on this branch against current main. Main already lowers the 40 LEA.HI and 17 SULD.1D programs, and the R2UR one is merged too. It would help to say which programs still fail on main and which hunk fixes each one.
Nits: on_big_stack is described as a precaution that turned out not to be needed, and it's Windows-only. Its trace path calls std::set_terminate from a worker thread, which replaces the handler for the whole process. The LOWER_DEBUG block and the RX_VK_LOWER_SKIP / RX_VK_TRACE_KEYS hooks are fine as diagnostics.
| uint64_t avail = 0; | ||
| if (!host->gpu_ptr(tex.gpu, &avail) || avail < total) return false; | ||
| const uint64_t guest = tex.gpu < (1ull << 40) ? tex.gpu : host->gpu_guest(tex.gpu); | ||
| if (!memory_guest_contained(guest, guest + total)) return false; |
There was a problem hiding this comment.
memory_guest_contained isn't declared or defined anywhere on this branch (it's in vk_internal.h / vk_memory.cpp on super-build/all), so vk_texture.cpp won't compile on its own.
| # here -- it gates -DRX_VK_STREAMLINE and an include path. | ||
| # (opennr_jobs: open neural rendering's sources, which the branch adds; without third_party/opendlss-nr, as in | ||
| # this fixture's tree, it adds none) | ||
| names = {"tree_headers", "vk_sources", "flag_text", "object_key", "streamline_sdk_path", "opennr_jobs"} |
| if (opcode == 0x593) return field < 3 ? 1 : 0; | ||
| // (super build) R2UR reads one GPR (bit 24); taken for a pair, OoT's R238 reached R239 (the uniform predicates) and a | ||
| // vertex program was not lowered | ||
| if (opcode == 0x3c2) return field == 1 ? 1 : 0; |
There was a problem hiding this comment.
Duplicate of #1806, which is now merged. Main has this line after the extended_renaming check, so please drop this one when you rebase.
| const u32 bit = bits[f], reg = insn.Bits(bit, 8); | ||
| // (super build) an immediate's low byte at bit 32 (LOP3 R236, R236, 0xfffffffd) was taken for R253: a | ||
| // scratch register the instruction never names, and two compute programs were not lowered | ||
| if (!is_register[f]) continue; |
There was a problem hiding this comment.
This skips every bit-32 field that isn't in the register_field list, and that list misses opcodes whose lowering reads a register at 32: TLD4 0xb63/0xf63, TMML 0xb69/0xf69, TEX 0xf60, SHFL.IDX 0x589 and STL 0x987. Before, those still got marked whenever another field set any_high. Now their Rb loses the REGMAP prefix. #1800 (approved) skips only bit 32 of form 4 of the ALU families, the actual immediate. Please keep that version and drop this one.
| coordinate > (dimension >= 4 ? 252U : dimension == 0 ? 254U : 253U)) | ||
| if ((insn.words[2] & ~0xf00U) != 0x001e0000 || channels == 0 || dimension > 5 || | ||
| binding >= 18 || offset > 0x1fff || insn.Bits(59, 2) || dest == 255 || | ||
| coordinate > (dimension == 2 || dimension >= 4 ? 252U : 253U)) |
There was a problem hiding this comment.
1D and 1D_BUFFER read one coordinate register, so R254 should stay allowed. Main allows it and test_lower_suld_1d.py checks for it. 1D_ARRAY reads two (x, layer), so its limit is 253. Accepting 1D_BUFFER and 1D_ARRAY also flips that test's refusal checks, and OoT doesn't seem to use either. Maybe leave them refused until a program needs them.
| const bool descriptor_address = (insn.words[2] & ~(0x3fU | 0xe00U)) == 0x0c1e1100; | ||
| if ((!descriptor_address && | ||
| (insn.words[2] & ~(0x04000000U | 0x3fU | 0xe00U | 0x2000U)) != 0x081e8100) || | ||
| (insn.words[2] & ~(0x04000000U | 0x3fU | 0xe00U | 0x6000U)) != 0x081e8100) || // (super build: bit 78 |
There was a problem hiding this comment.
Bits 77-79 all set is .STRONG.GPU (as on RED/ATOMG here), so dropping it loses the ordering. #1778 covers the same mask and fences the access with MEMBAR.GL when bit 78 is set, so I'd keep that version. This mask also lets 78 through without 77.
| // (super build) LEA.HI with a constant base: Rd = c[] + the high word of (Rc:Ra << s), bit 73 sign-extending Ra | ||
| // (Rc is RZ), .X (bit 74) adding the carry predicate in bits 87-90, as the register form below (0x211). | ||
| // OoT's compute programs compute addresses so (40 of them were not lowered) | ||
| if (insn.Bits(80, 1) && !insn.Bits(72, 1) && insn.Bits(81, 3) == 7 && !insn.Bits(62, 2) && |
There was a problem hiding this comment.
Main's mask also required bits 84-86 and 91-95 to be zero. This check doesn't test them, so an unknown qualifier there now passes silently. Could you keep the old mask, widened only by bit 74 (.X) and Rc? Also, test_lower_lea_hi_constant.py expects Rc≠RZ and a zero shift to be refused, so it fails now and needs updating.
| return finish(index); // the closing self-branch | ||
| // (super build) the closing self-branch of a program that only leaves through predicated EXITs (a loop whose | ||
| // @P0 EXITs end it: OoT's f24a3ebf686232fb), once nothing after it is a branch or call target | ||
| if (target == s64{index} * 16 && branch_predicate == (u64{7} << 16) && !main_exited) { |
There was a problem hiding this comment.
#1810 changes this same spot with a narrower rule (the previous instruction must be an unconditional BRA/EXIT). I think yours is sound, but only one should land. Small gaps compared with the EXIT path: no barrier_tokens check, and call targets ≤ index are allowed before main_exited. A test would help.
| // (super build) +0x68: the local memory a thread spills to (OoT's two largest compute programs, 255 registers: 3456 bytes). | ||
| // Taken as 0 before, their local loads named a variable that was never declared and the translation stopped | ||
| std::memcpy(&c.local_memory, ctl + 0x68, 4); | ||
| if (c.local_memory > (1u << 20)) c.local_memory = 0; // (not a size: as before) |
| // the fragment stage, where multiplying by FragCoord.w restores the screen-linear value.) | ||
| // Indexed fragment reads cannot undo that multiply, transform feedback would capture the | ||
| // scaled value, and mesh pipelines keep their existing per-primitive interpolation. | ||
| if (fs && !fs->program.info.loads_indexed_attributes && vrt.xfb_count == 0) { |
There was a problem hiding this comment.
This noperspective rewrite isn't mentioned in the description. It affects every VS+FS pipeline with a noperspective input on every vendor. Worth its own PR, or at least a line in the description.
…yal Family Tomb water), sirit names an undefined operand Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
6cf6178 to
fa5ee8c
Compare
|
Thanks for the thorough review; you were right on all counts. I rebuilt the branch on main at d7a7306, and it now builds and tests on its own. The description is rewritten. Blocking:
Overlap: since your review, #1806, #1800, #1778 and #1810 all merged, and main also has SULD.1D, LEA.HI with a constant base and ISBERD.O. I dropped all of ours in their favour, including the broad Not in the description before:
"70 programs": no longer claimed. Main now lowers everything those hunks did. Nits: |
Joey-Magnus
left a comment
There was a problem hiding this comment.
The rebase onto d7a7306 handled the earlier review, but main has moved again and vk_shader.cpp conflicts in translate_into. When you resolve it, note that #2042 removed oot_visual(), so the line you moved up with the fragment translation won't compile any more. Main's block now just translates and fills frt. The math is right: perspective interpolation of v*w, times FragCoord.w (1/w), gives back the screen-linear value. But it rewrites every VS+FS pipeline with a noperspective input on every vendor, and it hasn't run on AMD, the GPU it's for. Could you get one AMD run of the Royal Family Tomb with before/after screenshots, plus a quick NVIDIA check that nothing else with noperspective varyings moved? RX_VK_LOWER_SKIP isn't in translator_stamp, so programs translated while it was set stay unlowered in the cache after it's unset. That's fine for a diagnostic, but say so in the comment. The "(super build)" tags in comments can go.
|
This now conflicts with main in |
Steam Deck (RADV, VanGogh): the red particles and the red water both clearIndependent hardware confirmation that this fix addresses the red artifacts reported in #1634 / #1000, not just the Royal Family Tomb water. Setup. Hookshot Patch. This PR's noperspective work only: the six hunks in Before. Blinking red particle boxes over most of the screen, everywhere in Kokiri Village, and red-tinted sun highlights on the water — the same red as the particles. Reproduced consistently, and more obvious at 50% render resolution because upscaling magnifies the wrong pixels (a 1440p frame downscaled to the 800p panel averaged them away). After. Same build, same save, same settings, only the renderer swapped (original Mechanism, for the record. Main already emits Not tested here: geometry-shader and mesh-program paths, and any effect that reads indexed attributes (the PR leaves those alone by design). No game files, saves or logs attached. |
vk_shader.cpp: main (NotRexed#2042) removed the dead oot_visual workaround; the fragment translation this PR moves ahead of the vertex stage drops its call too.
ghost
left a comment
There was a problem hiding this comment.
Heads-up first: I merged main into the branch (d85bd1b) to clear the vk_shader.cpp conflict, so please pull before you push again. The conflict came from #2042 removing oot_visual() (Joey-Magnus pointed it out). The merge keeps your fragment translation moved ahead of the vertex stage and drops the oot_visual call. On my side the shader tests pass with it and the Vulkan back end builds.
Everything from my last review is handled. The rebuilt branch no longer touches lower_to_maxwell.h, vk_texture.cpp or the tests, so the build break, the three failing tests and the overlaps with #1806/#1800/#1778/#1810 are all gone, and so are on_big_stack and the set_terminate. The noperspective rewrite is now the PR's subject, and the description says what it changes on every vendor.
I went through the noperspective path again:
- The math holds, clipping included. Perspective-correct interpolation of v·w, times FragCoord.w (the interpolated 1/w), gives the screen-linear value. On a clipped edge, the clip-space lerp of v·w divided by the lerp of w is the same as the screen-space lerp of v.
- The vertex side covers every output element, including split variables (dedup by id). The fragment side only marks float, non-per-primitive inputs. The scaling runs after
ConvertDepthMode, which only touches z. - Indexed reads, XFB and the mesh path (its own
frt) are left alone. The GSEmitVertexhooks are a no-op for now, because only the VS+FS branch setsnoperspective_outputs. - No manual cache bump is needed.
HOOKSHOT_SHADER_STAMPhasheseden/,sirit/andvk_shader.cpp, so the new SPIR-V gets a new stamp by itself. - omgitsgela's Steam Deck run (RADV, VanGogh) is the AMD confirmation Joey asked for. With only the noperspective hunks applied, the red particles and the red water highlights from #1634/#1000 go away, and the same build is clean on an RTX 3090. Together with your NVIDIA run, that's enough for me.
Left, none of it blocking:
- Joey's point about
RX_VK_LOWER_SKIPstill stands: it isn't intranslator_stamp, so a bisection only works on programs that aren't cached yet. The easiest fix is to add"RX_VK_LOWER_SKIP"to the prefix list.bake_ds.cppneeds the same entry, becausetest_program_stamp_knobs.pychecks that the two lists match. Otherwise, say it in the comment. nx2/bake_ds/bake_ds.cppcopies this VS+FS path but doesn't setnoperspective_outputs/scaled_noperspective_inputs, so an offline-baked bank would still have the old interpolation. It's self-consistent and not broken, just without the fix. It already leaves out other live-only bits, so a follow-up is fine.- Nits: the
(super build)tags are still inemit_spirv.cpp,stream.handvk_shader.cpp. A before/after screenshot pair in the description would be good, since this shows on screen (CONTRIBUTING.md). omgitsgela's would do if they're OK with that.
CI: only the title check has run on this branch, and the build and test workflows haven't. That's why GitHub shows it as unstable.
| } | ||
| if (loads.AnyComponent(IR::Attribute::PositionX)) { | ||
| if (loads.AnyComponent(IR::Attribute::PositionX) || | ||
| info.passthrough.AnyComponent(IR::Attribute::PositionX) || noperspective_as_smooth) { |
There was a problem hiding this comment.
Nit: loads is already info.loads.mask | info.passthrough.mask (top of DefineInputs), so the info.passthrough.AnyComponent(PositionX) term repeats loads.AnyComponent(PositionX). || noperspective_as_smooth is the only part that does something.
|
Thanks, both of you, for merging main in and for the review. @omgitsgela: thank you for the Steam Deck run. Good to know it clears the red particles from #1634/#1000 too, not just the tomb water. The non-blocking notes are in #2782:
|
Port of upstream NotRexed#1830. OoT's Royal Family Tomb water writes clip.xy/clip.w as a noperspective varying; a clipped vertex with w near zero made AMD interpolate the wrong depth, whose exp2 overflowed into the red dotted line. The vertex stage now writes those outputs times Position.w, the fragment stage interpolates them perspective-correct and multiplies by FragCoord.w, and a geometry stage restores the unscaled values after EmitVertex. Indexed fragment reads, transform feedback and mesh pipelines keep the old path. Sirit's null Id now throws with the IR op that asked for it instead of aborting the game. tests/sirit_null_id_test.cpp checks that, and RX_VK_LOWER_SKIP leaves named programs unlowered for bisection.
NotRexed#1830 is ported with a sirit null-Id fixture; the headless RTX 4070 Ti boot ran 10,185 presents at 60 fps with 52,729 programs and no traps. The AMD water still needs AMD hardware.
On AMD, the water in the Royal Family Tomb shows a red dotted line. Separately, when sirit meets an undefined operand it aborts the whole game without logging why. This PR fixes both. It's part of the "super build" series, but it builds and passes the tests on its own, on current main.
super-build/shaders(#1830)super-build/upscaling-framegen(#1831)super-build/neural(#1833)super-build/core(#1834)What changed since the first review
The branch is rebuilt on main at d7a7306. Main now has its own versions of most of what this PR used to carry, so those parts are dropped:
MEMBAR.GLfences (fix(shaders): LDG/STG [URn+imm] with ordering bit 78 lower (2 compute programs) #1778).lower_to_maxwell.hand its tests are now exactly main's. Also dropped:on_big_stackand theRX_VK_TRACE_KEYStracing, with its process-wideset_terminate.vk_texture.cppchanges (memory_guest_contained, the unswizzle threshold,texture_touch_misses). They moved to feat: menu, renderer and build plumbing for #1831/#1833; upload hitches; lock diagnostics (builds with #1831, #1833) #1834, which describes them.test_shader_stamp_cache.pychange, which also moved to feat: menu, renderer and build plumbing for #1831/#1833; upload hitches; lock diagnostics (builds with #1831, #1833) #1834.What it does
vk_shader.cpp; Eden's SPIR-V back end:runtime_info.h,spirv_emit_context.*,emit_spirv_special.cpp,emit_spirv_context_get_set.cpp).noperspectivevarying. When a vertex with w close to zero is clipped, AMD interpolates the wrong depth, and itsexp2overflows into a red dotted line across the water.EmitVertex.noperspectiveinput, on every vendor. The result is the same value, interpolated a different way.Id) now throws instead ofstd::abort().EmitSPIRV's catch names the IR op and its arguments, and the program is reported as not translated instead of ending the game with nothing in the log.RX_VK_LOWER_SKIP=<key prefix,...>: leaves those programs unlowered, for bisecting. It's a diagnostic and does nothing unless set.Testing
nx2/build.py's renderer flags.tests/run_all.pyon this branch, locally (clang targeting MSVC, the CPM Vulkan headers, glslang 11.12), gives 324 passed. The 18 failures are the same 18 main at d7a7306 has on this machine, all down to this PC's toolchain, so there are no new failures.🤖 Generated with Claude Code