01 / values
Nonuniform Codebook10
More distinct levels sit near zero, where sharply peaked neural weights are concentrated.
The thesis
“Four-bit” describes the storage width—not the intelligence of the format.
Two 4-bit formats can spend identical weight bits yet reconstruct a model very differently. ROCmFP4 improves the quality per byte by reshaping the value grid, localizing scale error, optimizing reconstruction, and protecting sensitive tensors.
02 / metadata
Two local scales
The same two metadata bytes become one UE4M3 scale for each 16-weight half.
03 / fitting
MSE-optimal search
Finite scale candidates are scored by reconstruction error, with optional importance weighting.
04 / recipe
Tensor-aware STRIX
Large tolerant tensors stay compact while embeddings and attention K/V receive targeted protection.
| Property | Stock Q4_0 | Dual ROCmFP4 | ROCmFP4 FAST |
|---|---|---|---|
| Weights per block | 32 | 32 | 32 |
| Representable grid | Uniform integer steps | E2M1-derived Codebook10 | E2M1-derived Codebook10 |
| Scale metadata | 1 × FP16 (2 bytes) | 2 × unsigned E4M3 (2 bytes) | 1 × unsigned E4M3 (1 byte) |
| Scale grouping | 1 × 32 weights | 2 × 16 weights | 1 × 32 weights |
| Block size / density | 18 bytes / 4.50 BPW | 18 bytes / 4.50 BPW | 17 bytes / 4.25 BPW |
| Reference scale choice | Signed abs-max derived | Finite-scale MSE search | Finite-scale MSE search |
Precision note: this comparison uses stock Q4_0’s reference/no-imatrix path. llama.cpp can also use importance-weighted fitting for Q4_0 when an imatrix is supplied. ROCmFP4’s defining differences remain its codebook, scale encoding/layout, exhaustive native scale search, and STRIX tensor policy.
01
Smarter value grid
Put the detail where the weights live.
Stock Q4_0 lays down evenly spaced integers. ROCmFP4 uses 0, ±1, ±2, ±3, ±4, ±6, ±8, ±10—fine steps near zero, wider steps in the tails. Move the probe to see where each normalized grid snaps.
Normalized codebook explorer Interactive
Stock Q4_0 snap / absolute error3.0 / 0.30
ROCmFP4 snap / absolute error3.0 / 0.30
Normalized grid only; block-scale selection is intentionally isolated in the next two labs.
Codebook10
Dense center, economical tails
A sharply peaked distribution benefits from more nearby reconstruction choices around zero, while rare large values can tolerate coarser spacing.
0 · ±1 · ±2 · ±3 · ±4 · ±6 · ±8 · ±10
Retuned maximum
12 became 10—on purpose
The E2M1-derived outer magnitude was reduced after experiments on Qwen dense tensors, lowering the pull of rare outliers.
Exact identity
FP4-like, not native E2M1 math
The packed nibbles expand into signed integer levels for cheap integer dot products. This is not standard E2M1 data dispatched to an FP4 instruction.
02
Two scales, same cost
Keep one outlier from taxing all 32 weights.
A single Q4_0 scale couples the whole block. ROCmFP4’s two one-byte scales let each 16-weight half fit its own range. Increase the outlier and watch the calm half retain its detail.
32-weight reconstruction lab Interactive
outline = source · fill = reconstructed
1.025Q4_0 shared scale
0.813 / 0.094ROCmFP4 half scales
0.000Q4_0 block MSE
0.000ROCmFP4 block MSE
Local scales preserve the calm half. Calculating this block’s error…
Illustrative browser simulator. ROCmFP4 uses the implementation’s Codebook10 and full finite UE4M3 candidate set; the Q4_0 lane models its signed abs-max reference scale.
03
Best-fit scale search
Minimize total error—not outlier anxiety.
ROCmFP4 starts near max_abs / 10, expands across finite UE4M3 candidates, and scores reconstruction MSE. Safe pruning stops work that cannot beat the current best without changing the selected result.
Scale candidate error surface Linked to outlier
log scale → / log MSE ↑
Importance-weighted example
Treat the synthetic outlier as low-importance so the search may prefer the rest of the half-block.
Selected candidate
UE4M3 0x00
Scale details will appear here.
0.000best candidate MSE
0.000max-preserving candidate MSE
Why clipping can win
One imperfect value can buy 15 better ones
If a slightly smaller scale clips an unimportant outlier but sharply reduces error across the rest of the group, its total weighted MSE can be lower.
Quality guardrail
Restricted search was rejected
A faster candidate-window search worsened short WikiText-2 perplexity on the FAST path, so the exact exhaustive candidate set stayed.
04
Mixed precision where it matters
STRIX is a tensor policy, not a paint bucket.
The strongest ROCmFP4 artifacts do not force every tensor through one layout. They keep the largest tolerant tensors compact and spend more bits or scale granularity at sensitive boundaries.
STRIX_LEAN uses Q5_K at the token boundary, dual-scale ROCmFP4 for attention K/V, and 4.25-BPW FAST ROCmFP4 for most large dense tensors.
Token embeddings / protected
Embeddings are relatively small but coherence-sensitive. STRIX_LEAN promotes them to Q5_K; STRIX spends a little more with Q6_K.
Why fewer average bits can still mean better quality: uniform Q4 can over-spend on huge tolerant tensors while under-protecting a few smaller sensitive ones. STRIX moves the precision budget to the tensors where validation showed it mattered.
Technical boundaries
What ROCmFP4 is—and is not.
Clear boundaries keep a real engineering format from turning into a vague “FP4 is magic” claim.
E2M1-derived codebook
The stored values are expanded into signed integer lanes for dot products. They are not simply sent to a standard hardware FP4 instruction.
A family plus a recipe
Dual and FAST layouts solve different local problems, while STRIX selects tensor types according to sensitivity.
Better is workload-specific
The design targets quality per byte and AMD execution. Exact gains still depend on model family, tensor distribution, recipe, backend, and validation set.
The result
ROCmFP4 wins by making better decisions around the four bits.
The core idea is not “more precision.” It is better placement: a neural-shaped value grid, more local scales at equal metadata cost, an error-minimizing scale search, and mixed precision routed to the tensors that actually need it.
Design targetQ / BQuality per byte
Primary implementation sources
Read the format in the code.
The mechanics and caveats on this page were checked against the Ciru ROCmFPX implementation—not inferred from generic FP4 literature.