ROCm 7.2 + RDNA4: performance silently collapses (CK FMHA probe fails, ~2x slower)

[nlzy]

Summary

On ROCm 7.2.x with RDNA4 GPU. The build silently falls back to unfused attention.

With transformer nets this costs about half of NN throughput, with no warning at build time or runtime.

Environment

• KataGo v1.18.1, -DUSE_BACKEND=ROCM -DGPU_TARGETS=gfx1200
• RX 9060 XT 16 GB (gfx1200), ROCm 7.2.3, Debian GNU/Linux 13
• Model: kata1-tf3-b11c768-s11001M-d5973M.bin.gz

Performance

Same machine, same config (numSearchThreads=16, 500-visit benchmark):

Build	nnEvals/s
Probe failed (ROCm 7.2 headers)	204
CK FMHA (therock-7.13 headers)	434

The CK FMHA build uses -DKATAGO_CK_TILE_INCLUDE_DIR pointing at manually downloaded therock-7.13 headers.

Both paths pass testgpuerror -quick with equal error, so this is purely a performance loss.

Suggested fixes

1. Warn the user clearly at both build time and runtime.
   • Pros: trivial change; no behavior change, no code risk.
   • Cons: does not restore performance.
2. Fetch the required headers at CMake time (pinned rocm-libraries tag).
   • Pros: actually fixes the problem; no repo size growth.
   • Cons: build needs network access; depends on an external service.
3. Vendor the matching ck_tile headers into the KataGo repo.
   • Pros: actually fixes the problem; no network dependency.
   • Cons: repo grows (~26 MB of headers).

Happy to test any patch on gfx1200 / ROCm 7.2.

[lightvector]

Thanks. What's the best way to understand this issue? The difference is unusually large, yes, but how is this different than others case of "if you install an older version of a library, it might not perform as well as a more recent version". I think the same can sometimes be true with installing older versions of CUDNN or TensorRT, and so on.

[nlzy]

Thanks. This isn't different from other cases. The root cause is simply that the new ROCm library has a different API than the old one, so older libraries lose a lot performance, just like with old cuDNN or TensorRT. So this is not a KataGo defect, especially since KataGo already has `KATAGO_CK_TILE_INCLUDE_DIR` to specify the newer library. I'd just suggest keeping this issue as a reference — anyone hitting a performance drop on an older ROCm library can find the cause and the fix here. Feel free to close it; it'll still be searchable.

…

On Sun, Aug 30, 2026, 22:56 David J Wu ***@***.***> wrote: *lightvector* left a comment (lightvector/KataGo#1245) <#1245 (comment)> Thanks. What's the best way to understand this issue? The difference is unusually large, yes, but how is this different than others case of "if you install an older version of a library, it might not perform as well as a more recent version". I think the same can sometimes be true with installing older versions of CUDNN or TensorRT, and so on. — Reply to this email directly, view it on GitHub <#1245?email_source=notifications&email_token=ACZYBSMYYAZDBFOMD7XTJDT5MQ6A7A5CNFSNUABFM5UWIORPF5TWS5BNNB2WEL2JONZXKZKDN5WW2ZLOOQXTKNBWHE2DAOBWGM22M4TFMFZW63VGMF2XI2DPOKSWK5TFNZ2KYZTPN52GK4S7MNWGSY3L#issuecomment-5469408635>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ACZYBSOANBUHUIFIXGY5J4D5MQ6A7AVCNFSNUABFKJSXA33TNF2G64TZHMYTOMRWGQYDOMJRHNEXG43VMU5TKMRXGY2DKMRZGM22C5QC> . Triage notifications, keep track of coding agent tasks and review pull requests on the go with GitHub Mobile for iOS <https://github.com/notifications/mobile/ios/ACZYBSIURWGQX4ZFUYCYZFD5MQ6A7A5CNFSNUABFM5UWIORPF5TWS5BNNB2WEL2JONZXKZKDN5WW2ZLOOQXTKNBWHE2DAOBWGM22M4TFMFZW63VGMF2XI2DPOKSWK5TFNZ2KUZTPN52GK4S7NFXXG> and Android <https://github.com/notifications/mobile/android/ACZYBSL766AANOIP4OSYNS35MQ6A7A5CNFSNUABFM5UWIORPF5TWS5BNNB2WEL2JONZXKZKDN5WW2ZLOOQXTKNBWHE2DAOBWGM22M4TFMFZW63VGMF2XI2DPOKSWK5TFNZ2K4ZTPN52GK4S7MFXGI4TPNFSA>. Download it today! You are receiving this because you authored the thread.Message ID: ***@***.***>

[nlzy]

To be more specific: because the header layout/API differs between the new and old ROCm versions, KataGo can't probe CK at all with the old one, so it silently falls back to the fallback implementation. The CK path never gets enabled, which is why the performance gap is so large.

[Looong01]

I recommand upgrade to ROCm 7.14+

[nlzy]

I further tried the two prebuilt packages katago-v1.18.1-rocm7.2.4-linux-x64.zip and katago-v1.18.1-rocm7.14.0-linux-x64.zip.

Both run directly on my machine (environment described above). With the same settings, 7.14.0 achieves ~422 nnEvals/s while 7.2.4 only reaches ~187 nnEvals/s.

This doesn't surprise me. If I understand correctly, CK is a template library, so whether it can be used depends entirely on the headers available at compile time, not on the runtime environment.