[AaronStGeorge]

Overview

Closes ROCm/ggml-staging-automation#160.

Prefer shared row64 Q4_K/Q6_K weights for validated Qwen models while retaining the existing operation matchers. The immutable context flag is enabled only for LLM_ARCH_QWEN35; other architectures keep their normal selection policy. The tested opt-in files are Qwen3.8-27B, Qwen3.5-0.8B/2B/9B and MiniCPM-V-4.6's text backbone.

The change shares eligible weight representations across consumers and retains the Qwen short-prefill kernels and row64 Q4 metadata reuse. The fixes pass the logical row through the shared F32 reader, reuses known Q4/Q6 tile coordinates in prefill, and keeps the Q6 compiler hint local to prefill so decode code generation is preserved.

This PR includes the architecture-level row64 preference and all five fixes as separate commits. The results below measure this PR head, bd85479285423b239213519d0892c56d071864df.

Commit	Change
2cbd3dddc	Architecture-level row64 preference
826b09be9	pass logical row through shared F32 dequant reader
f3cd316c5	use the known row in Q6 tiled prefill
d36ffbf87	reuse Q6 row64 tile coordinates in tiled prefill
ea2c36bc1	reuse Q4 row64 tile coordinates in tiled prefill
bd8547928	keep the Q6 row-range hint local to tiled prefill

Final performance results

Baseline: frozen hrx-graph-develop-v2 at be335351871563a09520b0db4a63c21db2040807. Candidate: bd85479285423b239213519d0892c56d071864df. Hardware: Radeon RX 7900 XTX / gfx1100, with the same frozen HRX runtime on both sides. The current target branch has advanced beyond the measured baseline; these measurements do not claim validation of a rebased or merged result.

Standalone llama-bench, standard warmup enabled, full GPU offload, flash attention, F16 K/V cache, eight CPU threads, mmap, batch/microbatch 512. Prefill lengths 66/128/256/512/1024 and 128-token decode at prefix depth zero. Each invocation has five repetitions, ordered baseline/candidate/candidate/baseline (ABBA). All 1,800 samples are retained; the first five model results were reused unchanged when completing the remaining ten.

Numbers are percent throughput change, positive means faster. Each estimate is the geometric mean of the two adjacent candidate/baseline mean-throughput ratios. pp denotes prefill token count. These screening estimates do not establish statistical equivalence.

Model	Decode 128	pp66	pp128	pp256	pp512	pp1024
qwen3.8-27b	-0.23%	+28.09%	+26.90%	+0.06%	+0.38%	-0.43%
qwen3.5-0.8b	-0.40%	-1.26%	+3.10%	+0.09%	+0.46%	-0.73%
qwen3.5-2b	-0.57%	-0.77%	+0.58%	-0.55%	-0.51%	-0.43%
qwen3.5-9b	-0.32%	+9.59%	+13.63%	-0.06%	-0.12%	-0.20%
minicpm-v-4.6	-0.32%	+0.83%	+4.05%	-0.18%	+1.17%	-0.12%
phi-4-mini-instruct	-0.09%	-0.26%	-0.58%	+0.23%	+0.05%	+0.12%
gemma-4-12b	+0.18%	+0.02%	-0.05%	-0.21%	-0.03%	-0.09%
llama-3.1-8b	-0.09%	-0.24%	-0.34%	-0.26%	-0.15%	-0.18%
mistral-small-3.2-24b	-0.00%	+0.03%	-0.20%	+0.08%	+0.09%	+0.11%
minicpm5-1b	-0.30%	+1.06%	-0.43%	-0.73%	-0.81%	+0.01%
qwen3-1.7b	+0.73%	-0.19%	-0.69%	-1.93%	-1.26%	+0.02%
qwen3-30b-a3b-instruct-2507	-0.13%	-0.27%	-0.28%	-0.15%	-0.03%	+0.04%
qwen3-0.6b	+0.03%	+3.27%	-5.32%	-1.69%	-0.85%	-1.18%
lfm2-1.2b	+0.37%	-2.30%	+2.32%	-0.47%	+0.34%	+0.06%
qwen3.5-35b-a3b-mtp	-0.05%	+0.01%	+0.29%	-0.25%	+0.13%	+0.06%

Qwen27 and Qwen9 retain large 66/128-token gains. All decode estimates range from -0.57% to +0.73%. Remaining prefill losses include Qwen3-1.7B pp256/512 (-1.93%/-1.26%) and LFM2 pp66 (-2.30%). MiniCPM-V pp66 is unstable: its positive primary has mixed full pairs and negative later windows. Small differences are not assigned a causal explanation by this sweep.

Qwen3-0.6B stability check

Uneven early pp128 samples justified one separate 20-repetition ABBA check with the identical workload sequence. All 480 samples are retained, alongside the original screen.

Model / repetitions	Decode 128	pp66	pp128	pp256	pp512	pp1024
Qwen3-0.6B / r5 screen	+0.03%	+3.27%	-5.32%	-1.69%	-0.85%	-1.18%
Qwen3-0.6B / r20 followup	+0.27%	-1.03%	-0.54%	-0.63%	-0.05%	+0.10%

The -5.32% pp128 loss did not reproduce at that magnitude (-0.54% in the longer check); the initial pp66 gain also did not reproduce. Neither run replaces the other, and no samples were discarded.

Validation and scope

Independent source review found no blocking implementation findings in the retained fixes. The final clean committed build and four representative original controls passed. Inspected Q4/Q6 decode assembly and machine code match the shared-reader control, while inspected Q4/Q6 prefill code matches the preceding tile-coordinate implementation.

Known limitations remain: exact-Q6 and paired-Q4 strict-reference failures are inherited with identical parent/candidate outputs; tolerances were not changed. The generic packed-Q8/F32 diagnostic defect and native-weight materialization limitations are not repaired by these changes. The current sweep measures standalone throughput; it does not revalidate cold serving failures, speculative MTP, image inputs, or WebAssembly. The MTP-labelled MoE model uses ordinary decode here.

Requirements

• Contributing guidelines reviewed for this fork; human review and acceptance remain pending for this draft PR.
• AI usage disclosure: YES. Codex assisted implementation, validation, independent review and this description under the user's direction.