Skip to content

perf(fpx): fuse single-precision FMA, FRECPS and FRSQRTS on x86-64-v3 - #2383

Merged
hckrman101 merged 1 commit into
NotRexed:mainfrom
corentinbonjour:perf/fpx-fused-fma32
Nov 11, 2026
Merged

hckrman101 merged 1 commit into
NotRexed:mainfrom
corentinbonjour:perf/fpx-fused-fma32

Conversation

@corentinbonjour

Copy link
Copy Markdown
Contributor

On x86-64-v3 builds the single-precision FMADD/FMLA, FRECPS and FRSQRTS fast paths still compute in binary64 and round twice, although the CPU has a hardware FMA. One fused instruction gives the same results and makes FMA-bound guest code 22 to 45% faster.

What changed

suyu's runtime header uses __builtin_fmaf in recomp_fpx_fma32, recomp_fpx_recps32 and recomp_fpx_rsqrts32 only when RECOMP_FPX_HOST == 1 (an AArch64 host). On x86-64-v3, where RECOMP_FPX_FMA64 is already set from __FMA__, they took the binary64 form with a midpoint test, and fell back to the exact body on every binary32 midpoint.

New nx2/suyu_rt_fixes.py wraps a64_rtgen.runtime_h, as lift_fixes does for the lifter, and changes those three guards to RECOMP_FPX_HOST == 1 || RECOMP_FPX_FMA64. Nothing under third_party/ is edited. Each substitution must match exactly once, so a refreshed a64-kit that moves these lines fails loudly. nx2/lift.py and nx2/test_leaf.py install it next to lift_fixes.

Builds at x86-64-v2, baseline x86-64, MSVC and wasm are unchanged: RECOMP_FPX_FMA64 is 0 there. An existing game picks the change up on its next lift (a new file under nx2/ changes the lift key).

Measurements

llvm-mingw clang 23 from the v2.0.14 builder, at the build's lifted-code flags (-march=x86-64-v3 -ffp-contract=off -fno-strict-aliasing -O2), headers from the real generators. Synthetic kernels shaped like the lifted call forms; thread CPU cycles per element (QueryThreadCycleTime), median of 31 samples, 7 interleaved before/after rounds:

Kernel FZ off FZ on
FMA throughput 4.05 -> 2.79 (-31%) -31%
FMA dependent chain 19.43 -> 11.07 (-43%) -46%
4x4 mat*vec FMLA 30.78 -> 18.00 (-42%) -12% (noisy)
FRECPS + FMUL Newton x2 14.33 -> 10.16 (-29%) -25%
FRSQRTS Newton x2 21.90 -> 16.88 (-23%) -21%

The new header emits vfmadd/vfnmadd and no fmaf call.

The game-wide gain depends on how much of a frame runs these instructions, which needs an in-game frame-time comparison. I have not run one.

How it was tested

  • tests/fpx_sticky_test.c gets a fast mode: the inline fast paths against recomp_fpe_*, value and FPSR, under random FPCR/FPSR with a*b near 1, 2 or 3 as in the Newton steps, plus three binary32 midpoints that must be kept when the build has FMA. tests/test_fpx_sticky.py runs it with 8M cases in both the v3 and v2 builds: 0 mismatches. Without suyu_rt_fixes installed the v3 run fails (the midpoints fall back).
  • A separate fuzz of 200M cases per op (specials, subnormals, FZ on and off, cancellations, midpoints, Newton-like pairs): 0 mismatches. It catches sabotage: dropping the FZ check on z gave 85k mismatches in 2M.
  • python -I -B tests/test_fpx_sticky.py with HOOKSHOT_MINGW at the builder's llvm-mingw: passes.
  • python -I -B tests/run_all.py on Windows: the same failing set as untouched main at 32693f0, none new (the failures here come from tools missing on this PC). nx2/test_leaf.py needs the dump and was not run.

Notes

The only behaviour change is that binary32 midpoints now stay on the fast path, with the same result. #2324 moves the a64_rtgen.write call in nx2/lift.py; this change patches at import time, so it still applies, with at most a small conflict in the import block.

suyu's header takes the hardware FMA for recomp_fpx_fma32, _recps32 and
_rsqrts32 only on an AArch64 host. On x86-64-v3, which already has
RECOMP_FPX_FMA64, they computed in binary64 and fell back on every
binary32 midpoint. nx2/suyu_rt_fixes.py patches the generated
a64_suyu.h, as lift_fixes does for the lifter, so third_party is left
alone. tests/test_fpx_sticky.py checks the fast paths against the exact
bodies.

Assisted-by: Claude Opus 5.5
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

@hckrman101 hckrman101 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks right to me. I went through it against the Arm semantics:

  • The three #if guards now take the __builtin_fmaf branch that an AArch64 host already uses, so x86-64-v3 runs exactly the code path FPX1 was designed and proven for there. The keep conditions after it are unchanged: only a normal result (or a finite zero-product for FMADD) under an open gate (IXC already set, FPCR clean of RMode/DN/AHP/FZ16 and the inhibit bit) is kept, and FZ inputs and z are still screened. NaNs, infinities, tiny results and the FRECPS/FRSQRTS inf*0 case (NaN from the fma, never kept, exact body gives 2.0/1.5) all still fall back, so rounding mode, default-NaN, NaN propagation and flush-to-zero are all still the exact body's job.
  • FRSQRTS: fmaf(-a, b, 3) * 0.5f matches Arm's single rounding of (3 - a*b)/2 whenever the halved value is normal, which is the only case kept; overflow before halving gives inf, so it falls back.
  • Gating: RECOMP_FPX_FMA64 is only 1 for HOST 1, or HOST 2 with __FMA__ on GCC/clang, so x86-64-v2/baseline (host_march() on a CPU without FMA), MSVC and wasm keep the binary64 form. Since the game is built with the march of the PC it runs on, no FMA instruction can reach a CPU without one.
  • Same results on both platforms: CI ran test_fpx_sticky.py (with the new fast mode) and it passed on ubuntu-24.04 and windows-2022 (llvm-mingw). The midpoint cases I checked by hand (1+2^-12 squared, the FRECPS and FRSQRTS ones) really are binary32 midpoints, so the test does show the fused path is the one in use.
  • Cache: make_game.lift_key hashes every .py under nx2/, so the new file and the changed lift.py trigger a relift, and build.py's object key hashes the generated headers in gen/, so a64_suyu.h changing recompiles the chunks. No stale lifted code.
  • Patching a64_rtgen.runtime_h works because write() looks up runtime_h as a module global at call time, and the exactly-once substitution check is a good guard for a64-kit refreshes.

CI: the failing checks are test_vcore_fs_open / test_vcore_video_acquire (main fails them too right now) and the AppImage job timing out. None of that is from this PR.

What's missing is evidence from the game: one relift and run, to check it boots and plays normally, plus a before/after frame time from somewhere FMA-heavy, would back up the perf claim. That's not blocking given how much the test covers, but it would be good to have before merge. Nit: expect a small import-block conflict with #2324, as you noted.

@hckrman101
hckrman101 merged commit 8c52e7c into NotRexed:main Nov 11, 2026
8 of 13 checks passed
RufflezAU pushed a commit to RufflezAU/Hookshot that referenced this pull request Nov 12, 2026
…dedupe, residual-ratio interpolation

- vk_replay: NotRexed#1993 (batch out at an idle GPU) with RX_VK_BATCH_IDLE_MIN (default 2) so the
  GPU is fed during the frame instead of in one burst at its end: 20.4 -> 24.5 fps measured.
  Guest barriers no longer end the render pass (no pass-count change: harmless).
- vk_draw: the storage-pointer handle guard (upstream 758e563, fragment stages only) - the
  fairy and NPC boxes, the water flicker, objects vanishing.
- vk_draw: push-constant dedupe, index-buffer dedupe, and NOP_SET_RT only bumps the serial
  when the target set actually changed.
- fg_synth: the residual-ratio interpolation (SAFE=3, default): per pixel, the warp's error
  against a held frame's error, both measured on the same pixel; never a mix of two frames.
  SAFE=1 plain blend, SAFE=2 hold, SAFE=0 the old warp.
- REJECTED after measurement: NotRexed#2383 fused FMA (the game never leaves the title) and a sampler
  cache by id (same). Both reverted.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants