CIRU native vLLM
Official packed INT4 · ROCm · native K1 MTP39.67s
C5 · five simultaneous requests
vLLM is finished while llama.cpp still has 46.60 seconds left.
Native vLLM clears the complete burst at 39.67 seconds. AtomicChat llama.cpp does not finish until 86.28 seconds. The lead begins during prompt processing and persists through generation.
T+ 0.00s
8× replay
measured timeline ends at 86.28s
measured timeline ends at 86.28s
PREFILL · 2,048 prompt tokens
DECODE · 256 output tokens
DONE
AtomicChat llama.cpp
AD-IQ4_XXS GGUF · Atomic TurboQuant · Vulkan86.28s
BOTH STACKS LAUNCH AT T+0. Prompt processing begins on the same clock.
Visualization note: every request starts at T+0. TTFT and completion points are exact retained client timings. Prompt and output counters between those boundaries are interpolated for display; they are not a claim of per-token timestamp capture. Prompt counters show the nominal 2,048-token body; AtomicChat also processed 21 chat-template framing tokens per request.
Scaling, not a one-number stunt
What concurrency does to the queue
Direct ratios use the shared measured range, C2–C5. vLLM-only C1 and C6 points are shown as extensions; AtomicChat has no retained measured C1 or C6 row in this campaign.
Aggregate decode throughput
vLLM keeps gaining as requests are added; AtomicChat peaks at C4 and is flat at C5.
Whole-burst wall time
Lower is better. The separation grows from 30.53 seconds at C2 to 46.60 seconds at C5.
Prompt processing
Nominal 2,048-token prompt bodies divided by the launch-to-last-first-token window.
Observed stack multiplier
vLLM divided by Atomic for PP/TG; Atomic wall divided by vLLM wall for completed-burst throughput.
The C5 result
Concurrency turns the scheduler into the product
+185.5%More aggregate decode throughput: 60.55 vs 21.21 tok/s.
2.18×Nominal prompt throughput: 510.72 vs 234.40 tok/s.
46.60sMeasured time returned to the user before the Atomic C5 burst finishes.
The visual point is simple: single-stream TG does not tell you how a serving system feels when several agents, users, or tools arrive together. At C5, the native vLLM stack is already finished while the AtomicChat llama.cpp stack has more than half of its measured wall-clock run still ahead at the moment vLLM crosses the line.
Scope: this is a same-host, matched-shape comparison of two complete deployable stacks—not a controlled runtime-only A/B. The formats, endpoints, MTP path, context envelopes, and scheduler implementations differ. That is exactly what a user chooses when choosing a stack, but it prevents attributing the entire delta to one library in isolation.
Measured results
The full shared sweep
AtomicChat used one measured repetition per C2–C5 level after an excluded C5 warmup. Native vLLM values are three-repetition medians. Every measured request completed with the required output length.
| C | CIRU native vLLM | AtomicChat llama.cpp | Wall saved | ||||
|---|---|---|---|---|---|---|---|
| PP | TG | Wall | PP | TG | Wall | ||
| 2 | 520.98 | 38.23 | 20.86s | 266.61 | 14.06 | 51.39s | 30.53s |
| 3 | 514.93 | 46.17 | 28.08s | 241.96 | 19.39 | 62.98s | 34.90s |
| 4 | 523.95 | 57.12 | 32.91s | 240.19 | 21.28 | 73.46s | 40.55s |
| 5 | 510.72 | 60.55 | 39.67s | 234.40 | 21.21 | 86.28s | 46.60s |
Method and limits
Enough detail to audit the hype
CIRU native vLLM stack
- Official InclusionAI packed INT4 checkpoint; weights unchanged.
- ROCm native vLLM, TRITON_MLA, K1 native MTP.
- 256K model context, six active sequences, memory fraction 0.72.
max_num_batched_tokens=8192; completion endpoint.- Three measured repetitions C1–C6; medians reported.
AtomicChat llama.cpp stack
- AtomicChat AD-IQ4_XXS GGUF, Atomic TurboQuant build.
- Vulkan on RADV STRIX_HALO; flash attention and repacking on.
- 64K total context, five slots, f16/f16 KV.
- Chat endpoint; template adds 21 framing tokens per request.
- One measured repetition C2–C5; excluded C5 warmup.
Shared workload controls: simultaneous burst, nominal 2,048-token prompt body per request, exactly 256 forced output tokens, temperature 0, EOS ignored, distinct random fixtures, and no reusable prompt cache. Temperature 0 is a benchmark control, not InclusionAI's recommended live sampling preset.