Skip to content

rex3-jit: CIDMATCH probe used the fb_rgb base on aux-plane draws - #113

Merged
techomancer merged 1 commit into
techomancer:mainfrom
iblowmymind:crash-fix
Sep 18, 2026
Merged

techomancer merged 1 commit into
techomancer:mainfrom
iblowmymind:crash-fix

Conversation

@iblowmymind

Copy link
Copy Markdown
Contributor

The CID probe re-derived its byte offset from px_ptr against fb_rgb, which is only correct when the target plane is RGB/RGBA. For an OLAY/PUP/CID draw px_ptr is already fb_aux-based, so the probe read

fb_aux + (fb_aux - fb_rgb) + off

fb_rgb and fb_aux are independent Box<[u32]> allocations, so the delta is whatever the allocator chose — 700 MB and 584 MB in the two crashes seen — and the read lands outside both framebuffers. X11 draws menus, popups and the cursor into the overlay planes with CID checking live, which segfaulted the REX3-Processor thread after a few hours of IRIX 6.5. Both emitters carried the probe, so both were wrong.

Dm1::use_aux() now picks the base, and the probe reuses px_ptr directly when it is already an fb_aux pointer. The three inlined matches!(dm1.planes(), OLAY | PUP | CID) copies now call it too, so the predicate lives in one place.

Confirmed at the machine-code level for the shape that crashed (DRAW BLOCK, OLAY, CIDMATCH=4): the emitted shader loses the add/ldur/sub/ldr-reg-offset sequence from the crash report and issues a single ldr w, [x2, x, uxtw].

Every existing CIDMATCH test used DM1_RGB24_SRC, where the two bases cancel and the probe is right, which is why this survived. jit_cidmatch_aux_plane_matches_interp covers the other half: OLAY/PUP/CID x block/line adrmode x 3 masks x 4 CIDs against the interpreter, with a non-vacuity guard. Verified to fail without the fix. It fails there as a pixel mismatch rather than a SIGSEGV — in the test process the two framebuffer allocations are adjacent, so the wild read still hits mapped memory.

rules/rex3/cidmatch-aux-plane-base.md records the fix and the method for reading a crash report whose faulting frame is JIT code with no symbols.

The CID probe re-derived its byte offset from px_ptr against fb_rgb,
which is only correct when the target plane is RGB/RGBA. For an
OLAY/PUP/CID draw px_ptr is already fb_aux-based, so the probe read

    fb_aux + (fb_aux - fb_rgb) + off

fb_rgb and fb_aux are independent Box<[u32]> allocations, so the delta
is whatever the allocator chose — 700 MB and 584 MB in the two crashes
seen — and the read lands outside both framebuffers. X11 draws menus,
popups and the cursor into the overlay planes with CID checking live,
which segfaulted the REX3-Processor thread after a few hours of IRIX
6.5. Both emitters carried the probe, so both were wrong.

Dm1::use_aux() now picks the base, and the probe reuses px_ptr directly
when it is already an fb_aux pointer. The three inlined
matches!(dm1.planes(), OLAY | PUP | CID) copies now call it too, so the
predicate lives in one place.

Confirmed at the machine-code level for the shape that crashed (DRAW
BLOCK, OLAY, CIDMATCH=4): the emitted shader loses the
add/ldur/sub/ldr-reg-offset sequence from the crash report and issues a
single ldr w, [x2, x, uxtw].

Every existing CIDMATCH test used DM1_RGB24_SRC, where the two bases
cancel and the probe is right, which is why this survived.
jit_cidmatch_aux_plane_matches_interp covers the other half: OLAY/PUP/CID
x block/line adrmode x 3 masks x 4 CIDs against the interpreter, with a
non-vacuity guard. Verified to fail without the fix. It fails there as a
pixel mismatch rather than a SIGSEGV — in the test process the two
framebuffer allocations are adjacent, so the wild read still hits mapped
memory.

rules/rex3/cidmatch-aux-plane-base.md records the fix and the method for
reading a crash report whose faulting frame is JIT code with no symbols.
@techomancer
techomancer merged commit 44f4b24 into techomancer:main Sep 18, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants