Retired no longer in llama-swap.yaml — entry and/or weights removed after this verdict.
| Run | Score | tok/step | Log |
|---|---|---|---|
| No results-*.log found under this name. | |||
coder-x4 removed from the box 2026-07-27 ([[2026-07-27-model-cleanup]])
along with the base coder entry it depended on — grill round 5 caught
genuine tool-call loops on both. No model on the box currently serves
--parallel > 1 for chat, so this multi-slot throughput trick is currently
unavailable; revisit if a future loop-free model ships with parallel slots.
Everything below is history.
coder-x4 (alias gpt-oss-20b-4x32k, port 9115) is [[gpt-oss-20b]] served with
--parallel 4. Identical weights/sampler/KV to the coder entry; -c 131072 is
the TOTAL budget, so each slot gets 32768. Same VRAM (15.3GB), one 128k
window traded for four 32k ones.
Scaling, 500-token generations, identical prompt:
| concurrency | aggregate | per stream | speedup |
|---|---|---|---|
| 1 | 68.4 tok/s | 68.4 | 1.00x |
| 2 | 97.5 tok/s | 48.7 | 1.43x |
| 4 | 127.3 tok/s | 31.8 | 1.86x |
Sublinear — exactly what [[decode-bandwidth-ceiling]] predicts. **Batching is a
throughput lever, not a latency one:** 4 agents finish ~1.9x sooner in
aggregate, but each individual agent feels ~2.2x slower than running alone. If
one agent is waiting on the answer, run it solo on coder.
Batching does NOT degrade quality — verified, not assumed. Grill round 4 run
4-way concurrent, twice: 39/40. The one miss (clarify_missing: fabricated
a recipient instead of asking) was isolated by controls — 3 solo runs on the
same server scored 5/5 each, gpt-oss-20b's three historical grills only ever
failed expr_eval, and the second 4-way batch was a clean 20/20. So it is
sampling variance, not cross-slot contamination.
Full grill at 32k slots, 2026-07-26 (bench/results-coder-x4-20260726-140322.log):
21/23 — r1 6/8, r2 5/5, r3 5/5, r4 5/5, 17.3k tok / 4.7 min. The two misses are
expr_eval (gpt-oss-20b's documented weak spot, 3 passes in 7 across all runs)
and median_two_sorted (a second flaky task, ~2 in 9). CONTROLLED the same hour:
a 3-pass single-slot re-grill on the same build scored 23 / 21 / 23, so
coder-x4's 21/23 sits INSIDE the single-slot distribution — **the 32k window is
free**, not merely "no structural cost". Rounds 2-4 were 5/5 in every run either way.
Quote the yardstick as ~127 tok/s aggregate at 22/23 typical (22.3 mean), not 23/23.
Compare to [[bonsai-duo]], the other parallel entry: 2 slots x 128k, needs
the whole 32GB, and bonsai is 19/23. coder-x4 is 4 slots x 32k in half the
VRAM at 23/23 — prefer it unless an agent genuinely needs >32k of context or
vision.
This also exercised sync-models.sh's divide-by---parallel path correctly
(models.json advertises 32768, not 131072) — see [[sync-models-hook]].
Context for why this was run: it is the cheap yardstick for judging
[[nanbeige-4.2-3b]]-class small models for parallel agentic work, before
importing a vendor fork to serve one.