Qwen3.6 27B Vanilla MTP: Vulkan vs TheRock

Initial matched 4K / 128 speed matrix

PP is prompt processing. TG is generated-token throughput. TTFP includes uncached prompt ingestion. MTP accepted reports accepted draft tokens over proposed draft tokens. Higher PP/TG and lower TTFP are better. The UltraQuality row uses its latest matched 3,525-prompt / 128-generation cap-3 gate; the other rows retain their original matrix settings. The validated gates and cap selection below are the current tuning reference.

Validated new-TheRock and specialized-runner speed gates

These optimized-build gates are used alongside the Hard Mode audit. The stock lanes run stock llama.cpp; ROCmFP4 and UltraQuality run the required ROCmFPX or dedicated Vulkan implementations.

The dedicated Vulkan ROCmFP4 values are active counters from its earlier valid suite run and retain that lane's cap-4 profile. All other rows are standardized 3.5K-prompt / 128-generation gates.

Tool-Eval Hard Mode quality

The optional Hard Mode suite covers TC-70 through TC-84: 15 scenarios scored on tool choice, argument construction, recovery, state handling, and verification. Quality is intentionally reported separately from the short speed gate.

Comparable stored Hard Mode evidence

The intended-profile rows do not all share one sampler. They are useful cross-model context, not a claim that quantization alone caused every score difference.

Why MTP cap 3 replaced cap 6

The initial speed matrix used a six-token Q4 draft cap. A sweep on the same stock Q4/new-TheRock long request showed that six was over-drafting: acceptance fell to 59.04% and actual generation slowed.

Acceptance is a diagnostic, not the objective by itself. Cap 3 won because it delivered the highest measured generation throughput while clearing 80% acceptance.

Artifact footprint and effective bits per weight

Effective BPW is total GGUF bytes × 8 divided by 27,320,697,856 model parameters. It includes GGUF metadata and therefore measures the downloadable artifact, not merely the nominal tensor label. The run logs captured whole-host GTT on this unified-memory APU, not a clean per-process loaded allocation, so no misleading “VRAM loaded” column is reported. Runtime memory would also include KV cache, MTP context, and compute buffers and would vary with context and backend.

What the matrix shows

The backend winner depends on the quantization path. Vulkan leads generation for BF16, Q8, Q5, Q4, and UltraQuality; TheRock leads Q6 and ROCmFP4. TheRock consistently processes this prompt faster, but that PP advantage does not automatically translate into faster token generation.

UltraQuality is backend-stable in this harness. Matched Vulkan and new-TheRock runs both scored 60 with the same 8/2/5 scenario outcome and identical inference controls. TheRock processed the speed-gate prompt 57.3% faster, while Vulkan generated 6.2% faster; both accepted 90 of 110 MTP draft tokens.

At 60, UltraQuality matches the Q6 Hard Mode result, trails Q8 by seven points, and exceeds Q5 by three. The identical per-scenario backend outcome indicates that Vulkan versus new TheRock did not determine the measured tool-quality score.

The UltraQuality artifact is built from the same vanilla Unsloth BF16 source as the rest of the matrix using an importance matrix and the ranked leave-32 ROCmFPX/Q6_K-splice tensor policy. No fine-tuned derivative weights are included.

Method and scope