Crown Citadel GroupCiru Inference Labinvoice sandbox / Step3.7 FP3
Failure report · no contestant dashboard

Step3.7 FP3

Step 3.7 Flash ROCmFPX FP3 QualityPlus MTP was run through Pi against the invoice sandbox. The model did not complete the requested artifact.

No HTML dashboard was created. This page is the post-run benchmark report, not a replacement dashboard authored on the model’s behalf.
Wall time4m 58s
Tool calls48
PP179.30 t/s
TG34.29 t/s
Context262,144
Parsed rows112
Customer IDs0
Repeat calls29

What happened

  1. Workspace discoveryThe model inspected invoices, CRM, controller notes, bank exports, operational data, and representative trap PDFs.
  2. Batch parser createdIt wrote parse_invoices.js and generated 112 JSON records using the required page-one pdf-parse call.
  3. Extraction failedThe customer regex resolved zero customer IDs. The model began debugging the first invoice rather than repairing the batch logic directly.
  4. Stable tool loopIt issued the same parse command for INV-2025-1001_C011.pdf 29 times. The run was stopped at the reproducible loop boundary.
  5. Artifact contract missedNo HTML file existed at termination, so customer totals and trap accuracy were not scoreable.

Runtime evidence

llama.cpp processed 25,839 prompt tokens and 4,960 generated tokens. The request-weighted averages were 179.303 prompt tok/s and 34.290 generated tok/s. Pi recorded one tool error and no terminal assistant answer.

A separate preserved 64k control also failed after context overflow and automatic compaction. It is documented on the benchmark index and excluded from this scored run.

Result

Fast decode did not translate into task completion. The failure mode was agentic: inefficient exploration, a faulty regex, and repeated tool selection. No score is imputed for work that was never delivered.

Back to comparison