Rejected no longer in llama-swap.yaml — entry and/or weights removed after this verdict.
| Run | Score | tok/step | Log |
|---|---|---|---|
| No results-*.log found under this name. | |||
unsloth/Devstral-2-123B-Instruct-2512-GGUF:UD-IQ1_S, 28,445,876,448 B
(26.49 GiB). Dense 123B, Ministral3ForCausalLM: 88 layers, hidden 12288,
96 q-heads / 8 kv-heads GQA, head_dim 128, vocab 131072, 256k native ctx via
YaRN. The largest model ever run on this box, and the first at 1.x bpw. Entry
was added to llama-swap.yaml then removed after the grill; the GGUF is
still in the HF cache (26.49 GiB reclaimable via hf cache delete).
2/23, rounds 1-4 complete (r1 0/8, r2 0/5, r3 1/5, r4 1/5). Round 5 was
killed by the user after 3/6 tasks — all three 1800s timeouts. Compare the
27-35B Q4 entries on the same box, which all sit at 21-22/23. "Bigger model,
fewer bits" lost decisively to "smaller model, more bits".
Both "passes" are abstention tasks — tool_restraint (r3) and
clarify_missing (r4) are the two tasks whose correct move is to *not* call a
tool; it passed by emitting 3 and 45 tokens. Round 4's summary records
0 tool-steps on every task including the pass (tok/step = nan). Across
rounds 3-5 it never executed a single working tool call. Read the 2/23 as
functionally 0.
Probed all of this BEFORE grilling, which is what made the verdict safe:
| probe | result |
|---|---|
| add(a, b), 17 tok | perfect |
| prose | semantically right, word-corrupted ("represents four into a fixed array", the "scribed" location) |
| real coding task, temp 0 | invalid syntax: sorted sorted (, for i in range len (intervals), hallucinated self. in a plain function |
| tool call w/ schema | pure token salad: okat1Silke3S1442 Atkinson3S333 tiK4203 |
Greedy (temp 0) rules out sampling noise; the clean trivial case rules out a
template/detokenizer bug, which would have corrupted add(a,b) too. Quality
degrades with generation length and prompt complexity — the signature of
quantization damage. The tool-call path collapses hardest, which is why
tasks needing a real tool call ran away to the 16000-token cap and died on the
harness's 1800s client timeout instead of failing fast. At 10.3 t/s a
16000-token budget needs ~1600s, so those REQ-ERRs are part runaway, part
speed artifact — don't score them as clean runaways.
Honest limit: this box CANNOT A/B against a healthier quant of the same
weights (next size up for a 123B is ~45GB vs a 31GiB pool), so "IQ1_S is too
damaged" is not fully separable from "llama.cpp's Ministral3 path is subtly
wrong". Both fit the evidence. Arch support WAS confirmed present, so the model
card's "llama.cpp coming soon / may not be accurate" is stale for b10133
(ff067f76d): cd3c11890 (support Ministral3), 2fbe3b7bb (devstral-2
tool-call parser), 77078e80e (convert Ministral3 arch) are all ancestors of
the built HEAD — verify with git merge-base --is-ancestor, not
git log --all | grep, which also matches unmerged remote branches.
At -c 32768 with the default -ub 512 the model loads fine (294MiB /
604MiB free) and then dies on the first decode with a CUDA OOM in
ggml_cuda_pool_vmm::alloc. It is the transient compute pool, not the KV, that
doesn't fit — with vocab 131072 the logit buffer alone is vocab x ub x 4B =
~268MiB at ub 512. Largest config that survived decode: -c 24576 -b 1024
-ub 256, q4_0 KV, --tensor-split 50,50 → ~600-900MiB free per card, 10.3 t/s.
**Always send a real generation before trusting a context bump on a
memory-tight entry.** Cut -ub before cutting -c. KV at q4_0 here is
~99KiB/token (88 x 8 x 128 x 2 x 0.5625B); q8_0 KV (~191KiB/tok) doesn't fit at
any useful context, so this was a deliberate exception to
[[vram-spend-priority]].
Don't spend another download on a 1.x-bpw quant of a dense model without a
way to A/B it against a bigger quant. Prior art on this box already pointed the
same way — [[kimi-dev-72b]] REJECTED at IQ2_XXS, and
[[deepseek-r1-distill-llama-70b]] was tight at IQ2_XXS; IQ1_S is a full bit
below those. See [[gpu-device-ordering]] for the split mapping and
[[llama-swap-setup]] for the SIGHUP reload (which also unloads a removed entry).