Crown Citadel Group Ciru Inference Lab / Crown Citadel Group
Research synthesis / 2026-06-25

Why ROCmFP4 / ROCmFPX Runs Faster Than Stock Unsloth GGUF

The speedup is not one trick. It is a coupled serving stack: model artifacts use AMD-shaped packed formats, the runtime has kernels that consume those blocks directly, and Chadrock profiles tune MTP, KV cache, batch shape, and backend selection around the quant format.

+45.4% Q6_0_ROCMFPX_STRIX_QUALITY decode uplift at 64k vs Unsloth Q6.
53.25 tok/s Qwable ROCmFP4 dynamic n6 at 3,946 prompt tokens.
4.18x MTP n6 vs no-MTP control in the Qwable FP4 sweep.

Measured Evidence

Rows are sorted by decode uplift where there is a direct comparison. The class column separates clean model-format comparisons from served stack comparisons where MTP acceptance and request policy are part of the measured win.

local artifacts only
Comparison Class ROCmFP / Chadrock Baseline Decode Uplift Technical Reading
Qwable ROCmFP4 dynamic n6 vs vanilla Q4_K_M n4, 3,946 prompt Model + served MTP 53.25 tok/s 27.82 tok/s +91.4% FP4 artifact plus perfect draft acceptance on this prompt; not a pure kernel-only result.
ROCmFP4 STRIX_LEAN vs UD-Q5_K_XL, ROCm0 Model format + same MTP settings 27.60 tok/s 15.70 tok/s +75.8% Custom FP4 block layout and backend path beat a larger K-quant baseline in the same merged stack.
Qwable ROCmFP4 n4 vs vanilla Q4_K_M n4, 3,946 prompt Model + controlled draft depth 45.66 tok/s 27.82 tok/s +64.1% Same n_max removes the draft-depth explanation; remaining win is format, kernels, cache, and acceptance quality.
ROCmFP6 STRIX_QUALITY vs Unsloth Q6, 64k prompt Clean Unsloth source-line comparison 15.72 tok/s 10.81 tok/s +45.4% Long-context decode benefits from FPX kernels plus KV/cache/runtime decisions that hold up at 64k.
ROCmFP4 STRIX_LEAN vs UD-Q5_K_XL, Vulkan0 Model format + same MTP settings 24.90 tok/s 18.40 tok/s +35.3% Vulkan path also benefits, but less than ROCm0 for this profile.
ROCmFP6 STRIX_QUALITY vs Unsloth Q6, 512 prompt Clean Unsloth source-line comparison 29.52 tok/s 22.10 tok/s +33.6% Short-context decode win without relying on long-context stress behavior.
ROCmFP6 STRIX_QUALITY vs Unsloth Q6, 16k prompt Clean Unsloth source-line comparison 30.03 tok/s 25.76 tok/s +16.6% Middle-context row has the smallest uplift but still clears the baseline.
Research boundary: the largest rows are real user-facing served speedups, but they should not be described as isolated llama.cpp kernel wins. They include model format, MTP acceptance, cache policy, prompt length, backend selection, and request shape.

Model Improvements

These are properties of the GGUF artifact or quantization recipe. They travel with the model only when the runner understands the custom types and dispatches the intended backend paths.

artifact-owned

Real GGML Types

ROCmFP4 and ROCmFPX are not marketing aliases for stock Q4/Q6. They register separate GGML quantized types: Q4_0_ROCMFP4, Q4_0_ROCMFP4_FAST, Q3_0_ROCMFPX, Q6_0_ROCMFPX, and Q8_0_ROCMFPX.

AMD-Shaped Blocks

Blocks are built around compact integer codes multiplied by small floating scales. That reduces weight traffic and gives HIP/Vulkan kernels a regular decode path instead of generic K-quant unpacking.

Tensor-Aware Recipes

STRIX and STRIX_LEAN are not one blanket precision setting. Hot dense tensors can use FAST formats while quality-sensitive tensors stay protected or routed to higher precision.

Why this matters technically

  • The quant recipe changes which tensors become bandwidth-bound and which stay accuracy-bound.
  • Protected tensor routing explains why speed-oriented formats do not uniformly collapse structured output behavior.
  • Direct block decode avoids converting everything back through a generic scalar path before useful GPU work happens.
  • Model artifact size matters, but the more important point is that the format is aligned with the runtime's dot/dequant kernels.

llama.cpp / Chadrock Runner Optimizations

These are changes apart from the model artifact. Some help any Chadrock-served model; some only trigger when the model uses ROCmFP4/ROCmFPX tensor types.

runner-owned

Backend Kernel Coverage

  • HIP MMQ/MMVQ and vec-dot paths for ROCmFP4/FAST and ROCmFPX.
  • Vulkan shaders for dequant, matmul, copy, and fallback-safe reachability.
  • MUL_MAT_ID and MoE-sensitive routing so expert matmul avoids falling back to slow generic paths.

FlashAttention / KV Path

  • Format-aware K/V decode and dequant coverage.
  • Rows-per-thread and grouping choices selected by served throughput, not only microbenchmarks.
  • KV cache promotions where a requested low-precision cache would break coherency or speed.

Serving MTP Controls

  • Draft MTP exposed as a first-class served path, with request-level n_max, p_min, p_split, and draft-layer policy.
  • Dynamic profiles can raise draft depth on easy prompts and back off where acceptance deteriorates.
  • No-MTP controls show the magnitude: 12.78 tok/s vs roughly 52-53 tok/s in the Qwable sweep.

Regression Guards

  • Microbench improvements were not blindly accepted when sustained served decode regressed.
  • Guardrails rejected changes such as VDR settings that looked good in isolation but lost real Qwen MTP throughput.
  • This is why the runner optimizations are practical serving wins rather than only benchmark-kernel wins.
Answer to the narrow question: llama.cpp-only optimizations are the server request controls, draft-MTP scheduling, batch/ubatch/cache policy, backend selection, FlashAttention/KV handling, copy/dequant/matmul dispatch, and regression-guarded launch tuning. Custom model formats are not runner-only, but the kernels that execute those formats are runner changes.

Will This Help Any Chadrock Runner Model?

Yes for the generic serving path, conditionally for MTP-capable models, and only format-specific for ROCmFP4/ROCmFPX kernels. The transfer boundary is where many summaries become technically sloppy.

transfer matrix
Optimization Applies to Any Chadrock Model? Condition Expected Value
Server request handling, batching, slot policy, profile wiring Yes Model is served through the Chadrock runner. Lower overhead and better use of the same backend; effect size depends on prompt shape.
Draft-MTP scheduling and request controls Conditional Target has compatible MTP/drafter behavior and accepts draft tokens. Huge when acceptance is high; can regress when deeper drafts add rejected work.
KV cache policy and FlashAttention profile tuning Mostly Backend and cache type must be supported for the model architecture and context length. Most visible at long context or when a low-precision KV request would hurt coherency.
HIP/Vulkan ROCmFP4 and ROCmFPX matmul/dequant kernels No Requires tensors encoded as ROCmFP4, ROCmFP4_FAST, or ROCmFPX types. Large, but format-specific; a stock Q4_K/Q6_K file will not hit these kernels.
Tensor-aware STRIX recipes and protected-tensor routing No Requires a model converted with those quantization recipes. Moves the speed/quality frontier for that artifact; it is not a generic runner effect.
Practical rule: a random model using Chadrock runner benefits from runtime polish. A model using Chadrock runner plus ROCmFP4/ROCmFPX tensors benefits from both runtime polish and the custom packed-format kernels. A model with strong MTP acceptance benefits from the full stack.

Source Trail

These are the local files behind the report. The HTML is a visual companion to the markdown report, not a replacement for the code audit notes.

reproducibility
Primary report
/srv/desktop-data/cirudata/research/rocmfp4-rocmfpx-speed-technical-report-2026-06-25.md
ROCmFP6 vs Q6
/home/crown/bench-results/llama/rocmfp6-quality-vs-q6-speed-20260624/SUMMARY.md and /home/crown/bench-results/llama/rocmfp6-quality-rocm-vs-vulkan-20260624/SUMMARY.md
Qwable FP4 vs Q4
/home/crown/bench-results/llama/qwable-5-27b-chadrock-v2-rocmfp4-vs-vanilla-q4-20260621/SUMMARY.md
MTP sweep
/home/crown/bench-results/llama/qwable5-27b-coder-rocmfpx-fp4-mtp-sweep-20260621/SUMMARY.md and /home/crown/research/rocmfpx-pr-drafts-20260620/ROCmFPX/docs/ROCmFP4-MTP-COMPARISON.md
Repos audited
/home/crown/research/rocmfpx-pr-drafts-20260620/ROCmFPX and /home/crown/research/rocmfp4-llama-pr-drafts-20260620/rocmfp4-llama