Chadrock ROCmFP4 vs Unsloth Upstream Q4/Q6 Comparison
Date: 2026-06-30.
This adds the requested comparison rows against the ROCmFP4 Chadrock builds and Unsloth Q4/Q6. Unsloth rows used upstream llama.cpp, not the ROCmFPX/CHADROCK fork, version 1 (7c082bc). Chadrock ROCmFP4 rows used profile-capable CHADROCK/ROCmFPX runners because upstream llama.cpp cannot load those ROCmFP4 GGUFs.
Harness: isolated llama-server, one slot, /completion streaming API, llama_benchmark.py api, prompt cache disabled, EOS ignored, gen=128, non-wiki 10KB and 20KB deterministic prompts from the prior recipe sweep. Reported context tokens are actual tokenized prompt lengths: about 1977 and 3946.
Memory note: memory columns are normalized to one displayed Peak total RAM GiB value instead of separate VRAM/GTT/system RAM triples.
Side-by-Side ECharts Comparison
The dashboard below is generated from the existing Headline 4k MTP Ranking table in this artifact. Speed is decode TG. Q4-ish rows are compared only against Unsloth Qwen3.6 27B UD-Q4_K_XL; Q6-ish rows are compared only against Unsloth Qwen3.6 27B UD-Q6_K_XL.
Speed vs BPW, 4k MTP
X = TG decode speed. Y = tensor BPW. Each point is one headline 4k MTP row.
+/- Speed vs Relevant Unsloth Baseline
Q4-ish means BPW below 6.0 and uses the Q4 baseline. Q6-ish means BPW 6.0 or higher and uses the Q6 baseline.
Headline 4k MTP Ranking
| Rank | Model / recipe | Source | BPW | Quality evidence | Runner/backend | PP | TG | Draft | Peak VRAM/GTT/RAM GiB |
|---|---|---|---|---|---|---|---|---|---|
| 1 | Qwable Chadrock v2 ROCmFP4 | new comparison | 4.3355 | PPL: none found; HumanEval 159/164, HE+ 155/164 | CHADROCK Vulkan | 298.48 | 51.26 | 108/108 | 1.96/16.18/28.88 |
| 2 | Unsloth Qwen3.6 27B UD-Q4_K_XL | new comparison | 5.2409 | PPL: none found locally | upstream llama.cpp Vulkan | 263.88 | 41.60 | 108/112 | 1.99/16.36/29.36 |
| 3 | Crown FP4 Fast | current recipe sweep | 4.8194 | PPL 6.1562 +/- 0.08400 | ROCmFPX rocm | 291.76 | 41.23 | 108/108 | 0.59/0.27/35.01 |
| 4 | Crown FP4 Balanced | current recipe sweep | 5.3039 | PPL 6.1192 +/- 0.08276 | ROCmFPX rocm | 254.12 | 39.69 | 108/108 | 0.59/0.27/36.62 |
| 5 | Qwen3.6 27B Chadrock ROCmFP4 STRIX_LEAN | new comparison | 4.3355 | PPL: none found; HumanEval 159/164, HE+ 155/164, BCHard 46/148 | CHADROCK Vulkan | 292.54 | 39.24 | 100/104 | 1.76/16.89/29.82 |
| 6 | chadrock3.6 27B pi-agent ROCmFP4 | new comparison | 4.3355 | PPL: none found; HumanEval 157/164, HE+ 151/164, BCHard 40/148 | CHADROCK ROCm0 | 307.23 | 33.26 | 95/95 | 0.59/0.27/30.35 |
| 7 | Crown FPX-Q6 Quality | current recipe sweep | 7.6163 | KLD-slice PPL 7.3142; full PPL not found | ROCmFPX rocm | 229.82 | 29.41 | 108/108 | 0.59/0.27/40.74 |
| 8 | Crown FPX-Q6 Balanced | current recipe sweep | 7.6146 | PPL 6.5212 +/- 0.09323; KLD-slice PPL 7.3049 | ROCmFPX rocm | 223.04 | 29.29 | 108/109 | 0.59/0.27/40.56 |
| 9 | Unsloth Qwen3.6 27B UD-Q6_K_XL | new comparison | 7.6146 | PPL 6.5296 +/- 0.09339; HumanEval 160/164, HE+ 153/164, MBPP+ 295/378 | upstream llama.cpp Vulkan | 198.38 | 23.76 | 106/123 | 1.93/23.37/36.40 |
| 10 | Crown FP6 Legacy | current recipe sweep | 7.3779 | PPL 6.5097 +/- 0.09282; KLD-slice PPL 7.3162 | ROCmFPX vulkan | 187.13 | 20.03 | 105/132 | 1.98/25.58/37.94 |
New Comparison Rows
| Model | Quant | BPW | Runner/backend | Ctx | Mode | PP | TG | Draft | Peak VRAM/GTT/RAM GiB | Quality evidence |
|---|---|---|---|---|---|---|---|---|---|---|
| chadrock3.6 27B pi-agent ROCmFP4 | Q4_0_ROCMFP4_STRIX_LEAN | 4.3355 | CHADROCK ROCm0 | 2048 | mtp | 299.96 | 34.54 | 94/97 | 0.59/0.27/29.41 | PPL: none found; HumanEval 157/164, HE+ 151/164, BCHard 40/148 |
| chadrock3.6 27B pi-agent ROCmFP4 | Q4_0_ROCMFP4_STRIX_LEAN | 4.3355 | CHADROCK ROCm0 | 2048 | no-mtp | 357.57 | 13.45 | 0.59/0.24/27.04 | PPL: none found; HumanEval 157/164, HE+ 151/164, BCHard 40/148 | |
| chadrock3.6 27B pi-agent ROCmFP4 | Q4_0_ROCMFP4_STRIX_LEAN | 4.3355 | CHADROCK ROCm0 | 4096 | mtp | 307.23 | 33.26 | 95/95 | 0.59/0.27/30.35 | PPL: none found; HumanEval 157/164, HE+ 151/164, BCHard 40/148 |
| chadrock3.6 27B pi-agent ROCmFP4 | Q4_0_ROCMFP4_STRIX_LEAN | 4.3355 | CHADROCK ROCm0 | 4096 | no-mtp | 359.84 | 13.27 | 0.59/0.24/27.21 | PPL: none found; HumanEval 157/164, HE+ 151/164, BCHard 40/148 | |
| Qwable Chadrock v2 ROCmFP4 | Q4_0_ROCMFP4_STRIX_LEAN | 4.3355 | CHADROCK Vulkan | 2048 | mtp | 263.41 | 50.23 | 107/116 | 1.94/15.47/29.50 | PPL: none found; HumanEval 159/164, HE+ 155/164 |
| Qwable Chadrock v2 ROCmFP4 | Q4_0_ROCMFP4_STRIX_LEAN | 4.3355 | CHADROCK Vulkan | 2048 | no-mtp | 310.02 | 14.29 | 1.85/12.73/26.96 | PPL: none found; HumanEval 159/164, HE+ 155/164 | |
| Qwable Chadrock v2 ROCmFP4 | Q4_0_ROCMFP4_STRIX_LEAN | 4.3355 | CHADROCK Vulkan | 4096 | mtp | 298.48 | 51.26 | 108/108 | 1.96/16.18/28.88 | PPL: none found; HumanEval 159/164, HE+ 155/164 |
| Qwable Chadrock v2 ROCmFP4 | Q4_0_ROCMFP4_STRIX_LEAN | 4.3355 | CHADROCK Vulkan | 4096 | no-mtp | 318.98 | 14.23 | 1.87/12.85/25.29 | PPL: none found; HumanEval 159/164, HE+ 155/164 | |
| Qwen3.6 27B Chadrock ROCmFP4 STRIX_LEAN | Q4_0_ROCMFP4_STRIX_LEAN | 4.3355 | CHADROCK Vulkan | 2048 | mtp | 168.33 | 39.72 | 100/105 | 1.99/15.72/29.50 | PPL: none found; HumanEval 159/164, HE+ 155/164, BCHard 46/148 |
| Qwen3.6 27B Chadrock ROCmFP4 STRIX_LEAN | Q4_0_ROCMFP4_STRIX_LEAN | 4.3355 | CHADROCK Vulkan | 2048 | no-mtp | 296.54 | 14.04 | 1.98/13.19/26.01 | PPL: none found; HumanEval 159/164, HE+ 155/164, BCHard 46/148 | |
| Qwen3.6 27B Chadrock ROCmFP4 STRIX_LEAN | Q4_0_ROCMFP4_STRIX_LEAN | 4.3355 | CHADROCK Vulkan | 4096 | mtp | 292.54 | 39.24 | 100/104 | 1.76/16.89/29.82 | PPL: none found; HumanEval 159/164, HE+ 155/164, BCHard 46/148 |
| Qwen3.6 27B Chadrock ROCmFP4 STRIX_LEAN | Q4_0_ROCMFP4_STRIX_LEAN | 4.3355 | CHADROCK Vulkan | 4096 | no-mtp | 326.94 | 14.04 | 1.87/13.67/26.49 | PPL: none found; HumanEval 159/164, HE+ 155/164, BCHard 46/148 | |
| Unsloth Qwen3.6 27B UD-Q4_K_XL | UD-Q4_K_XL | 5.2409 | upstream llama.cpp Vulkan | 2048 | mtp | 220.92 | 30.14 | 101/151 | 1.97/16.23/29.33 | PPL: none found locally |
| Unsloth Qwen3.6 27B UD-Q4_K_XL | UD-Q4_K_XL | 5.2409 | upstream llama.cpp Vulkan | 2048 | no-mtp | 257.44 | 12.10 | 1.94/15.15/28.79 | PPL: none found locally | |
| Unsloth Qwen3.6 27B UD-Q4_K_XL | UD-Q4_K_XL | 5.2409 | upstream llama.cpp Vulkan | 4096 | mtp | 263.88 | 41.60 | 108/112 | 1.99/16.36/29.36 | PPL: none found locally |
| Unsloth Qwen3.6 27B UD-Q4_K_XL | UD-Q4_K_XL | 5.2409 | upstream llama.cpp Vulkan | 4096 | no-mtp | 282.43 | 12.05 | 1.94/15.29/28.02 | PPL: none found locally | |
| Unsloth Qwen3.6 27B UD-Q6_K_XL | UD-Q6_K_XL | 7.6146 | upstream llama.cpp Vulkan | 2048 | mtp | 135.18 | 23.41 | 107/116 | 1.95/23.23/36.12 | PPL 6.5296 +/- 0.09339; HumanEval 160/164, HE+ 153/164, MBPP+ 295/378 |
| Unsloth Qwen3.6 27B UD-Q6_K_XL | UD-Q6_K_XL | 7.6146 | upstream llama.cpp Vulkan | 2048 | no-mtp | 162.52 | 8.31 | 1.88/22.19/35.90 | PPL 6.5296 +/- 0.09339; HumanEval 160/164, HE+ 153/164, MBPP+ 295/378 | |
| Unsloth Qwen3.6 27B UD-Q6_K_XL | UD-Q6_K_XL | 7.6146 | upstream llama.cpp Vulkan | 4096 | mtp | 198.38 | 23.76 | 106/123 | 1.93/23.37/36.40 | PPL 6.5296 +/- 0.09339; HumanEval 160/164, HE+ 153/164, MBPP+ 295/378 |
| Unsloth Qwen3.6 27B UD-Q6_K_XL | UD-Q6_K_XL | 7.6146 | upstream llama.cpp Vulkan | 4096 | no-mtp | 194.11 | 8.28 | 1.99/22.19/35.02 | PPL 6.5296 +/- 0.09339; HumanEval 160/164, HE+ 153/164, MBPP+ 295/378 |
4k MTP vs No-MTP Uplift, New Rows
| Model | BPW | MTP TG | No-MTP TG | Uplift | MTP PP | No-MTP PP |
|---|---|---|---|---|---|---|
| Qwable Chadrock v2 ROCmFP4 | 4.3355 | 51.26 | 14.23 | 3.60x | 298.48 | 318.98 |
| Unsloth Qwen3.6 27B UD-Q4_K_XL | 5.2409 | 41.60 | 12.05 | 3.45x | 263.88 | 282.43 |
| Unsloth Qwen3.6 27B UD-Q6_K_XL | 7.6146 | 23.76 | 8.28 | 2.87x | 198.38 | 194.11 |
| Qwen3.6 27B Chadrock ROCmFP4 STRIX_LEAN | 4.3355 | 39.24 | 14.04 | 2.79x | 292.54 | 326.94 |
| chadrock3.6 27B pi-agent ROCmFP4 | 4.3355 | 33.26 | 13.27 | 2.51x | 307.23 | 359.84 |
Current Recipe Reference Rows
These are from the already-completed recipe ROCm/Vulkan 2k/4k sweep, included here so the new Chadrock/Unsloth rows can be read against the current recipe set without hunting through a second file.
| Recipe | Quant/recipe | BPW | Runner/backend | Ctx | Mode | PP | TG | Draft | Peak VRAM/GTT/RAM GiB | Quality evidence |
|---|---|---|---|---|---|---|---|---|---|---|
| Crown FP4 Balanced | Q6_0_ROCMFPX_STRIX_LEAN | 5.3039 | ROCmFPX rocm | 2048 | mtp | 260.39 | 36.10 | 106/121 | 0.59/0.27/36.05 | PPL 6.1192 +/- 0.08276 |
| Crown FP4 Balanced | Q6_0_ROCMFPX_STRIX_LEAN | 5.3039 | ROCmFPX rocm | 2048 | no-mtp | 303.62 | 11.32 | 0.59/0.24/29.98 | PPL 6.1192 +/- 0.08276 | |
| Crown FP4 Balanced | Q6_0_ROCMFPX_STRIX_LEAN | 5.3039 | ROCmFPX rocm | 4096 | mtp | 254.12 | 39.69 | 108/108 | 0.59/0.27/36.62 | PPL 6.1192 +/- 0.08276 |
| Crown FP4 Balanced | Q6_0_ROCMFPX_STRIX_LEAN | 5.3039 | ROCmFPX rocm | 4096 | no-mtp | 308.86 | 11.18 | 0.59/0.24/30.08 | PPL 6.1192 +/- 0.08276 | |
| Crown FP4 Balanced | Q6_0_ROCMFPX_STRIX_LEAN | 5.3039 | ROCmFPX vulkan | 2048 | mtp | 226.14 | 23.04 | 107/116 | 1.96/18.51/30.80 | PPL 6.1192 +/- 0.08276 |
| Crown FP4 Balanced | Q6_0_ROCMFPX_STRIX_LEAN | 5.3039 | ROCmFPX vulkan | 2048 | no-mtp | 243.11 | 7.32 | 1.88/15.76/27.98 | PPL 6.1192 +/- 0.08276 | |
| Crown FP4 Balanced | Q6_0_ROCMFPX_STRIX_LEAN | 5.3039 | ROCmFPX vulkan | 4096 | mtp | 228.09 | 23.99 | 108/108 | 1.99/19.23/31.57 | PPL 6.1192 +/- 0.08276 |
| Crown FP4 Balanced | Q6_0_ROCMFPX_STRIX_LEAN | 5.3039 | ROCmFPX vulkan | 4096 | no-mtp | 247.72 | 7.31 | 1.89/15.90/28.11 | PPL 6.1192 +/- 0.08276 | |
| Crown FP4 Fast | Q6_0_ROCMFPX_STRIX_SPEED | 4.8194 | ROCmFPX rocm | 2048 | mtp | 285.65 | 38.58 | 106/121 | 0.59/0.27/34.37 | PPL 6.1562 +/- 0.08400 |
| Crown FP4 Fast | Q6_0_ROCMFPX_STRIX_SPEED | 4.8194 | ROCmFPX rocm | 2048 | no-mtp | 337.68 | 12.35 | 0.59/0.24/28.56 | PPL 6.1562 +/- 0.08400 | |
| Crown FP4 Fast | Q6_0_ROCMFPX_STRIX_SPEED | 4.8194 | ROCmFPX rocm | 4096 | mtp | 291.76 | 41.23 | 108/108 | 0.59/0.27/35.01 | PPL 6.1562 +/- 0.08400 |
| Crown FP4 Fast | Q6_0_ROCMFPX_STRIX_SPEED | 4.8194 | ROCmFPX rocm | 4096 | no-mtp | 330.33 | 12.21 | 0.59/0.24/28.50 | PPL 6.1562 +/- 0.08400 | |
| Crown FP4 Fast | Q6_0_ROCMFPX_STRIX_SPEED | 4.8194 | ROCmFPX vulkan | 2048 | mtp | 253.86 | 28.21 | 107/116 | 1.91/17.02/29.49 | PPL 6.1562 +/- 0.08400 |
| Crown FP4 Fast | Q6_0_ROCMFPX_STRIX_SPEED | 4.8194 | ROCmFPX vulkan | 2048 | no-mtp | 278.20 | 9.60 | 1.98/14.13/26.52 | PPL 6.1562 +/- 0.08400 | |
| Crown FP4 Fast | Q6_0_ROCMFPX_STRIX_SPEED | 4.8194 | ROCmFPX vulkan | 4096 | mtp | 257.06 | 29.38 | 108/108 | 1.93/17.75/30.10 | PPL 6.1562 +/- 0.08400 |
| Crown FP4 Fast | Q6_0_ROCMFPX_STRIX_SPEED | 4.8194 | ROCmFPX vulkan | 4096 | no-mtp | 282.61 | 9.57 | 1.97/14.27/26.53 | PPL 6.1562 +/- 0.08400 | |
| Crown FP6 Legacy | Q6_0_ROCMFPX_STRIX_QUALITY | 7.3779 | ROCmFPX rocm | 2048 | mtp | 137.29 | 9.62 | 61/380 | 0.59/0.27/42.38 | PPL 6.5097 +/- 0.09282; KLD-slice PPL 7.3162 |
| Crown FP6 Legacy | Q6_0_ROCMFPX_STRIX_QUALITY | 7.3779 | ROCmFPX rocm | 2048 | no-mtp | 262.46 | 8.59 | 0.59/0.24/36.42 | PPL 6.5097 +/- 0.09282; KLD-slice PPL 7.3162 | |
| Crown FP6 Legacy | Q6_0_ROCMFPX_STRIX_QUALITY | 7.3779 | ROCmFPX rocm | 4096 | mtp | 208.74 | 11.03 | 70/331 | 0.59/0.27/43.10 | PPL 6.5097 +/- 0.09282; KLD-slice PPL 7.3162 |
| Crown FP6 Legacy | Q6_0_ROCMFPX_STRIX_QUALITY | 7.3779 | ROCmFPX rocm | 4096 | no-mtp | 264.94 | 8.51 | 0.59/0.24/36.42 | PPL 6.5097 +/- 0.09282; KLD-slice PPL 7.3162 | |
| Crown FP6 Legacy | Q6_0_ROCMFPX_STRIX_QUALITY | 7.3779 | ROCmFPX vulkan | 2048 | mtp | 185.24 | 20.53 | 105/126 | 1.97/24.83/37.21 | PPL 6.5097 +/- 0.09282; KLD-slice PPL 7.3162 |
| Crown FP6 Legacy | Q6_0_ROCMFPX_STRIX_QUALITY | 7.3779 | ROCmFPX vulkan | 2048 | no-mtp | 190.37 | 5.47 | 1.92/22.06/34.34 | PPL 6.5097 +/- 0.09282; KLD-slice PPL 7.3162 | |
| Crown FP6 Legacy | Q6_0_ROCMFPX_STRIX_QUALITY | 7.3779 | ROCmFPX vulkan | 4096 | mtp | 187.13 | 20.03 | 105/132 | 1.98/25.58/37.94 | PPL 6.5097 +/- 0.09282; KLD-slice PPL 7.3162 |
| Crown FP6 Legacy | Q6_0_ROCMFPX_STRIX_QUALITY | 7.3779 | ROCmFPX vulkan | 4096 | no-mtp | 194.09 | 5.47 | 1.92/22.20/34.48 | PPL 6.5097 +/- 0.09282; KLD-slice PPL 7.3162 | |
| Crown FPX-Q6 Balanced | ranked leave-32 splice | 7.6146 | ROCmFPX rocm | 2048 | mtp | 175.74 | 26.00 | 106/121 | 0.59/0.27/39.78 | PPL 6.5212 +/- 0.09323; KLD-slice PPL 7.3049 |
| Crown FPX-Q6 Balanced | ranked leave-32 splice | 7.6146 | ROCmFPX rocm | 2048 | no-mtp | 248.32 | 8.32 | 0.59/0.24/36.84 | PPL 6.5212 +/- 0.09323; KLD-slice PPL 7.3049 | |
| Crown FPX-Q6 Balanced | ranked leave-32 splice | 7.6146 | ROCmFPX rocm | 4096 | mtp | 223.04 | 29.29 | 108/109 | 0.59/0.27/40.56 | PPL 6.5212 +/- 0.09323; KLD-slice PPL 7.3049 |
| Crown FPX-Q6 Balanced | ranked leave-32 splice | 7.6146 | ROCmFPX rocm | 4096 | no-mtp | 257.41 | 8.26 | 0.59/0.24/36.99 | PPL 6.5212 +/- 0.09323; KLD-slice PPL 7.3049 | |
| Crown FPX-Q6 Balanced | ranked leave-32 splice | 7.6146 | ROCmFPX vulkan | 2048 | mtp | 184.21 | 24.06 | 107/116 | 1.92/25.60/37.90 | PPL 6.5212 +/- 0.09323; KLD-slice PPL 7.3049 |
| Crown FPX-Q6 Balanced | ranked leave-32 splice | 7.6146 | ROCmFPX vulkan | 2048 | no-mtp | 193.39 | 7.87 | 1.99/22.71/35.39 | PPL 6.5212 +/- 0.09323; KLD-slice PPL 7.3049 | |
| Crown FPX-Q6 Balanced | ranked leave-32 splice | 7.6146 | ROCmFPX vulkan | 4096 | mtp | 186.52 | 22.78 | 106/123 | 1.96/26.32/38.71 | PPL 6.5212 +/- 0.09323; KLD-slice PPL 7.3049 |
| Crown FPX-Q6 Balanced | ranked leave-32 splice | 7.6146 | ROCmFPX vulkan | 4096 | no-mtp | 193.55 | 7.88 | 1.99/22.84/35.10 | PPL 6.5212 +/- 0.09323; KLD-slice PPL 7.3049 | |
| Crown FPX-Q6 Quality | ranked leave-16 splice | 7.6163 | ROCmFPX rocm | 2048 | mtp | 208.05 | 25.46 | 106/121 | 0.59/0.27/39.79 | KLD-slice PPL 7.3142; full PPL not found |
| Crown FPX-Q6 Quality | ranked leave-16 splice | 7.6163 | ROCmFPX rocm | 2048 | no-mtp | 254.52 | 8.32 | 0.59/0.24/36.85 | KLD-slice PPL 7.3142; full PPL not found | |
| Crown FPX-Q6 Quality | ranked leave-16 splice | 7.6163 | ROCmFPX rocm | 4096 | mtp | 229.82 | 29.41 | 108/108 | 0.59/0.27/40.74 | KLD-slice PPL 7.3142; full PPL not found |
| Crown FPX-Q6 Quality | ranked leave-16 splice | 7.6163 | ROCmFPX rocm | 4096 | no-mtp | 261.35 | 8.26 | 0.59/0.24/37.05 | KLD-slice PPL 7.3142; full PPL not found | |
| Crown FPX-Q6 Quality | ranked leave-16 splice | 7.6163 | ROCmFPX vulkan | 2048 | mtp | 188.99 | 24.42 | 107/116 | 1.92/25.60/37.97 | KLD-slice PPL 7.3142; full PPL not found |
| Crown FPX-Q6 Quality | ranked leave-16 splice | 7.6163 | ROCmFPX vulkan | 2048 | no-mtp | 198.71 | 8.11 | 1.99/22.71/35.00 | KLD-slice PPL 7.3142; full PPL not found | |
| Crown FPX-Q6 Quality | ranked leave-16 splice | 7.6163 | ROCmFPX vulkan | 4096 | mtp | 189.26 | 23.11 | 106/123 | 1.96/26.32/38.72 | KLD-slice PPL 7.3142; full PPL not found |
| Crown FPX-Q6 Quality | ranked leave-16 splice | 7.6163 | ROCmFPX vulkan | 4096 | no-mtp | 197.35 | 8.11 | 1.99/22.85/35.12 | KLD-slice PPL 7.3142; full PPL not found |
Takeaways
- Fastest 4k MTP row in this combined local evidence set is
Qwable Chadrock v2 ROCmFP4at298.48 PP / 51.26 TG, despite only4.3355tensor BPW. - The best upstream
llama.cppUnsloth baseline isUD-Q4_K_XLat263.88 PP / 41.60 TGat 4k MTP. It is larger than Chadrock ROCmFP4 at5.2409BPW and still trails Qwable Chadrock v2 in decode. UD-Q6_K_XLis the quality/PPL anchor among the new baselines (6.5296 +/- 0.09339) but is not a speed competitor here:198.38 PP / 23.76 TGat 4k MTP,8.28 TGwithout MTP, and about23.37 GiBpeak GTT.- MTP is mandatory for these MTP models if speed matters. At 4k, new-row decode uplifts ranged from
2.51xto3.60xversus no-MTP. - No local PPL evidence was found for the Chadrock ROCmFP4 or Unsloth Q4 rows; the available quality evidence for those rows is coding benchmark output rather than Wikitext PPL.
Artifacts
- New comparison evidence bundle: retained in the internal benchmark archive.
- New comparison result store: retained in the internal benchmark archive.
- New comparison structured result log: retained in the internal benchmark archive.
- New comparison replay script: retained in the internal benchmark archive.
- Existing recipe reference report: retained in the internal benchmark archive.
- Existing recipe evidence bundle: retained in the internal benchmark archive.
Verification: llama_benchmark.py verify-store returned ok: true, rows: 20 for the new comparison run, and the script ended with no llama-server processes remaining.
Source materials and benchmark stores are retained in the internal benchmark archive.