coder-agentic

coder-agentic-quant-ladder (coder-agentic-quant-ladder.md)

Ladder grill 2026-08-24, bench/grill-ladder-20260824-2057/ (n=3 reps, same base

weights Qwen3-Coder-Next 80B-A3B, same harness, split/batch/sampler held IDENTICAL so

quant + KV + context are the only variables). Three rungs, the box's most

aggressively quantized model taken up the precision ladder as far as 2×16 GB allows.

| suite (3 reps) | coder-agentic IQ2_XXS (2b, q4_0 KV, 262k, ~72 t/s) | -hq IQ2_M (2b, q8_0 KV, 262k, 62 t/s) | -q3 IQ3_XXS (3b, q8_0 KV, 131k, 49.8 t/s STALE — now ~67, see below) |

|---|---|---|---|

| coding r1-r4 (/69) | 66/69 (22/23) | 66/69 | 66/69 |

| R5 complete | 5/6,6/6,6/6 | 6/6,6/6,6/6 | 5/6,5/6,4/6 |

| R5 looped | 1,0,0 (1 total) | 0,0,0 (0) | 1,1,2 (4 total) |

| klayout unaided (/24) | 4 | 5 | 3 |

| klayout+ref (/24) | 16 | 15 | 18 (+2) |

| office unaided (/27) | 2 | 2 | 12 (+10) |

| office+ref (/27) | 12 | 14 | 18 (+6) |

| realcase (/3) | 2 | 2 | 2 |

| envelope | clean | clean | clean |

Three verdicts

IQ2_M ("hq", a better 2-bit + q8_0 KV): BOUGHT NOTHING — deleted. Identical to

baseline on every suite (coding 66/69, klayout+ref 5/8, office 0/9) at 14% slower

decode. Weights deleted too; ssk500 was at 97%. A better 2-bit is not a better

model here.

IQ3_XXS ("q3", the only 3-bit that fits): a TRADE, not an improvement.

• GAINS on the reference-driven suites: office unaided 2->12 (+10), office+ref

12->18 (+6), klayout+ref 16->18 (+2). More bits genuinely let it execute API

details (the openpyxl / KLayout off-by-one bugs the 2-bit gets wrong).

• REGRESSES R5 long-horizon: looping 1->4 across the 3 reps, looping in EVERY

rep (1/1/2, escalating), complete 6/6->4/6. This is the SAME axis that retired

[[gpt-oss-20b]] and demoted [[mellum2-12b-a25b-trial]] / mellum2-instruct.

• COSTS half the context (131k vs 262k). The "30% slower" half of this is NO LONGER TRUE — see the speed section at the end.

4-bit and up: do not fit. [[no-4bit-coder-next-fits]] — smallest 4-bit build is

31.8 GiB vs ~30 usable on 2x16 GB (weights alone, no KV); BF16 is 160 GB,

impossible. 3-bit is the ceiling for "better weight" on this hardware.

The reusable finding

**Quant precision and long-horizon loop-resistance move in OPPOSITE directions on

this model.** More bits gave it the precision to nail isolated API-detail tasks

(office, KLayout+ref) but made it LOOP on multi-step agentic work (R5). A model

that is "better" on the discrimination suites is NOT better for agentic driving —

and R5 is the dealbreaker axis on this box ([[grill-round5-agentic-loops]]).

⚠️ CORRECTED 2026-08-26 — see the KV-isolation follow-up below. The ladder

compared 2-bit (q4_0 KV) vs 3-bit (q8_0 KV) — TWO variables. The looping turned

out to be the q8_0 KV, not the 3-bit weight bits. The "more bits → more loops"

framing was a confound.

This is the second confirmation that bit-depth is not where the *decisive*

capability lives ([[coder-agentic-default-driver]] found REAP Q6 scored below Q4

and ternary bonsai kept perfect vision). But it refines it: more bits DO help on

narrow reference-driven tasks — they just cost you the long-horizon axis and half

your context. Pick the rung by use case, not by "best quant":

• office / KLayout-with-a-reference task -> coder-agentic-q3 (3-bit). Kept on

the NVMe for exactly this; measurably better there.

• agentic / long-horizon / driver -> coder-agentic (2-bit) stays the default.

Clean R5, 262k, 72 t/s. The 3-bit's R5 looping disqualifies it as a driver.

Note: the 2-bit baseline had ONE R5 loop in rep1 here (1/6,0/6,0/6) — the

[[coder-agentic-regrilled]] n=2 saw 0/6 twice; R5 is a sample

([[round5-is-a-sample-not-a-measurement]]). The 3-bit's 4 loops across 3 reps

(1,1,2) looked like a directional regression — **but see the follow-up: it was the

q8_0 KV, not the 3-bit weights.**

KV-ISOLATION FOLLOW-UP (2026-08-26) — the looping is the q8_0 KV, not the weights

Two follow-up grills (same 3-bit IQ3_XXS weights, same 32000-budget protocol):

A. q4_0 KV + 262k: LOADED fine on static arithmetic (~1.2 GiB spare, same as

q8_0+131k) but the child OOM-crashed 6× on the large-prompt klayout/office

requests (every rep, 0 klayout/office measured). Context-length runtime overhead

(compute buffer / full-attention scratch, which scales with -c, NOT with KV

precision) breached the spare under a real request — [[kv-sizing-full-attention-layers]]:

"compute buffer binds before KV." The entry comment's knife-edge warning ("raise -c

only after watching a real long-context run, never on arithmetic alone") was right;

static KV arithmetic undercounts it. So **q8_0 KV + 131k is load-bearing for

STABILITY, not just context** — the 262k window is only safe with the smaller 2-bit

weights (the 2-bit baseline runs q4_0 KV + 262k stably).

B. q4_0 KV + 131k (clean apples-to-apples with the q8_0+131k original):

| 3-bit q3, 131k, n=3 | q8_0 KV (original) | q4_0 KV (clean run) |

|---|---|---|

| R5 loops | 4 (1,1,2) | 1 (1,0,0) |

| office+ref | 18/27 | 14/27 (−4) |

| klayout+ref | 18/24 | 18/24 (same) |

| coding / realcase / envelope | 22/23 / 2/3 / clean | same |

| speed (r1 decode) | ~61 tok/s | ~61 tok/s (same) |

• Looping = the q8_0 KV. q4_0 KV cuts loops 4→1 (back to the 2-bit baseline

level). The 3-bit weights with q4_0 KV loop only 1×. The 262k run's "3 loops"

was confounded by the 262k context adding its own; the matched 131k run is the

truth. The "more bits → more loops" framing above was wrong — it was the KV.

• KLayout+ref (18/24) = the 3-bit WEIGHTS (q4_0 KV keeps it; q8_0 KV not

needed). Office+ref (18/27) needs BOTH — 3-bit weights give 14/27 (> 2-bit's

12/27), q8_0 KV tops it to 18/27. So q8_0 KV is a double-edged sword: **+4

office+ref AND +3 loops.**

• q4_0 KV gives NO speed gain (~61 tok/s both) — the 26.5 GiB of 3-bit weights

dominate per-token bandwidth, not the ~1.5 GiB KV; halving the KV saves ~2-3%,

lost in noise. Confirms [[vram-spend-priority]]: q8 KV is "free" on speed here.

VERDICT: kept q8_0 KV + 131k (reverted). q4_0 KV trades 3 loops for 4 office+ref

at no speed gain and the same KLayout+ref — not worth it for a specialist whose role

is office/KLayout-with-ref. So the corrected pick: **q8_0 KV + 131k for the

office+ref specialist (accept its 4 loops — it's not a driver); 2-bit + q4_0 KV

+ 262k stays the clean-R5 driver.**

The yaml coder-agentic-q3 comment said "UNGRILLED" at write time — written before

this ladder finished. The rung WAS grilled here; this file is its verdict.

SPEED FIGURES RECONCILED 2026-09-09 — the 49.8 t/s above is STALE

Three different decode numbers for coder-agentic-q3 were in circulation, in this

file and in llama-swap.yaml. **They are all real; they differ by measurement shape

and by SPLIT MODE**, and only the last pair describes what the entry runs today:

| figure | when | split | shape |

|---|---|---|---|

| 49.8 t/s | 2026-08-24 ladder | layer | ladder protocol — STALE, superseded |

| ~61 t/s | 2026-08-26 KV-isolation | layer | in-suite r1 (matches the 61.5 below) |

| 64.4 -> 72.5 t/s | 2026-09-02 A/B | layer -> tensor | clean microbenchmark, n=3 warm |

| 61.5 -> 67.3 t/s | 2026-09-02 A/B | layer -> tensor | in-suite, across a real grill |

The entry has run -sm tensor since 2026-09-02 ([[sm-tensor-rejects-tensor-split]]),

so the current numbers are ~72 t/s microbenchmark and ~67 t/s in-suite. Quote the

in-suite figure for planning: the microbenchmark's +12.5% becomes +9.4% in a real

suite, because tensor mode is 7.4% SLOWER on prefill and suites mix prefill-heavy turns.

**Consequence for the ladder verdict above: the "3-bit costs 30% speed" argument is

dead.** At ~67 in-suite against the 2-bit's ~72 the rungs are near parity on decode,

so choose between them on R5 looping and context (131k vs 262k), not on speed.

Caveat, do not over-read the parity: the 2-bit coder-agentic is still on

--tensor-split 48,52 (layer) and has NEVER been measured under -sm tensor, so it

may gain too — and its ~72 comes from a different protocol than the 67.3. The honest

claim is "the 3-bit's decode penalty is now small or gone", not "they are equal".

coder-agentic-regrilled (coder-agentic-regrilled.md)

Run 2026-08-09, vllm/grill-coder-agentic-20260809.sh, 9 minutes, 18 suite

runs, 0 failed. Same protocol and same jsonl as

[[regrill-20260808-four-model-trust]], driven through llama-swap on :8090.

First time this model has been on the grill_round*.py instrument at all — its

recorded 31.0/33 came from metric_sweep.py on a different 33-task suite, and

it had never been scored on KLayout or office.

| suite | rep1 | rep2 | combined | field position |

|---|---|---|---|---|

| R1-R4 | 21/23 | 22/23 | 43/46 | 3rd (both fables 44/46) |

| R5 agentic | 6/6 | 6/6 | 12/12 | tied best, 0/6 looped twice |

| klayout from memory | 0/8 | 0/8 | 0/16 | joint-worst (only fable-fusion is non-zero) |

| klayout + API ref | 6/8 | 7/8 | 13/16 | ties glm-flash; +5 over qwen3-coder |

| office | 4/6 | 0/6 | 4/12 | WORST in the table |

| office + ref | 4/6 | 2/6 | 6/12 | worst |

| cap-hits | 0 | 0 | 0 in 18 runs | |

| throughput | | | 70 tok/s | |

IT CLOSES THE KLAYOUT REGRESSION. [[qwen3-coder-regrilled]] records the

promotion as "a deliberate KLayout regression" — 8/16 with the reference, worst

of five — and tells you to select glm-flash-awq by name for KLayout codegen.

coder-agentic is 13/16, level with glm-flash, while also scoring 43/46 coding

against glm-flash's 42 and qwen3-coder's 41. On those two axes it strictly

dominates both.

LEANNESS DOES NOT BUY WALL CLOCK — the old sweep's headline is misleading.

Median coding output 270 tok (max 931), the leanest ever measured here, beating

qwen3-coder's 323 and glm-flash's 2564. But it runs at **70 tok/s vs

qwen3-coder's 141, and the identical 18-suite set took 9 min vs 7**. Token

economy and wall clock are different axes; [[coder-agentic-default-driver]]

measured only the first and read it as speed. A 2-bit 80B-A3B is lean per task

and slow per token.

Its "ZERO spread" does not survive this harness. That came from 3 repeats of

one suite; across 9 suites x 2 it totals 8 — the joint-highest in the table with

fable-fusion. Another entry for [[single-run-scores-are-unreliable]]: stability

measured on one suite does not generalise to others.

Office is the real reject risk. rep1 4/6 then rep2 0/6 — the largest

swing in the dataset. Not a harness artifact: all three rep2 failures are the

same genuine bug, an openpyxl Reference that starts at the first data row

while also passing titles_from_data=True, so the first data point is eaten as

the series name and every chart is off by one row (chart plots rows [(3,7)],

expected 2-7). It writes that correctly some runs and not others.

**HARNESS DEFECT FOUND AND FIXED: the office denominator was

outcome-dependent.** A failed struct stage skipped its dependent pdf stage

and the summary divided by graded-only, so failing early SHRANK the denominator

and partially hid itself — this model's 0/3 was really 0 of 6. The same defect

was then found in the closed-loop chain. Both fixed the same day; all numbers on

this page are POST-fix. See [[office-loop-denominator-fix]].

Practical upshot: a genuine driver contender, NOT an automatic promotion.

Against the incumbent qwen3-coder it wins coding (+2), KLayout (+5), context

(262k vs 200k, with 3/3 needle at 237k) and token economy; it loses wall clock

(9 vs 7 min on the same work) and office (4/12 vs 8/12). Text-only permanently

(Qwen3NextForCausalLM, no vision_config, no mmproj in any repo), so it

brings no vision either way. Left UNPROMOTED pending that call — clients

reference the qwen3-coder id in five places

([[vllm-office-server-client-wiring]]).

REAL-WORK CONFIRMATION 2026-08-12 (Giovanni, unprompted): coder-agentic

successfully coded a KLayout task that gemma-awq could not. This is a rare

real-work corroboration of a bench column — the recorded 13/16 KLayout+ref, noted

here as "closes qwen3-coder's KLayout regression", is exactly the gap

[[gemma-4-26b-awq-vllm]] documents in the driver. Weigh this against the demotion:

the demotion was decided on RETRIEVAL (gemma 248k vs its 237k) and wall clock, NOT

on KLayout, and KLayout is where it beats the driver on real work. Per

[[grill-does-not-validate-real-use]] a real-work signal outranks a suite column —

so reach for coder-agentic over gemma-awq for KLayout work, regardless of

which is nominally the driver.

CONTROLLED A/B vs qwen3next-instruct — 3 batteries each, 2026-08-12

Same base (Qwen3-Next 80B-A3B), CODER tune vs INSTRUCT tune, quant-matched

(both unsloth UD-IQ2_XXS) with a byte-identical serving config, so weights were the

only variable. Trial premise: coder-agentic's office 4/8 is the worst on the box and

an Instruct tune should fix it.

| suite | coder-agentic | qwen3next-instruct |

|---|---|---|

| 23-task | 22, 22, 20 (21.3, spread 2) | 22, 22, 22 (spread 0) |

| R5 | 6/6 x3, 0 loops | 5/6 x3, 0 loops |

| KLayout unaided | 0, 0, 1 (0.3) | 3, 4, 1 (2.7) |

| KLayout +ref | 7, 7, 6 (6.7) | 5, 4, 4 (4.3) |

| office unaided | 0, 0, 0 | 4, 4, 2 (3.3) |

| office +ref | 6, 6, 6 — MAX every time | 6, 4, 4 (4.7) |

| real-case gate | 1/3 | 3/3 |

| tok/step | 55.1 | 57-59 |

VERDICT: KEEP coder-agentic, REJECT qwen3next-instruct. The Instruct tune gives

up 6.7 -> 4.3 on assisted KLayout — the capability actually used — and 6/6 -> 5/6

on R5, in exchange for a coding mean 0.7 higher with zero spread. Bad trade.

THE TRIAL PREMISE WAS FALSE. The Instruct tune DOES fix the unaided office

column (3.3 vs 0.0) — but that weakness only exists WITHOUT a reference, and with

one coder-agentic scores the ceiling in all three batteries. **The archive's

"coder-agentic has the WORST office on the box (4/8)" is its UNAIDED number and is

misleading**: given a reference it is perfect. Its real property is being

reference-driven, and this box always loads skills.

THE REFERENCE-DRIVEN vs MEMORY-DRIVEN SPLIT is the reusable finding. Measured on

three independent suites:

• coder-agentic: near-zero from memory (KLayout 0.3, office 0.0), excellent with a

ref (KLayout 6.7, office at ceiling). Gain +6.4.

• qwen3next-instruct: moderate from memory, barely moves with a ref. Gain +1.6.

So the klayout-classes skill is worth ~+6 to the Coder tune and ~+1.6 to the

Instruct tune. **A workflow built on handing a model an API reference is wasted on

the Instruct tune** — the opposite profile from everything else here.

LEANNESS IS A BASE PROPERTY, NOT A FINE-TUNE ONE: 55.1 vs 57-59 tok/step, 9 tool

steps each, within ~5% on every efficiency measure. Both tunes inherit it.

THE REAL-CASE GATE IS ~40% FOR BOTH — the most decision-relevant result here

The A/B's realcase column (1/3 vs 3/3) looked decisive, so it was re-run **10x per

model** (~25 s/run):

• coder-agentic 3/10, qwen3next-instruct 5/10
• Fisher exact p = 0.65 (10 each), p = 0.24 including the batteries (8/13 vs

4/13). NO established difference. The 3/3-vs-1/3 was noise.

• 20 runs: 8 PASS, 6 NO_GDS, 3 GEOMETRY, 2 API_RECALL, 1 SPEC_PARSE.

**Giovanni's actual Excel->GDS task completes ~40% of the time on either model, and

fails four different ways.** That is far more important than the model choice: expect

to verify and iterate on real KLayout+Excel work regardless of which model runs it.

It is also a concrete instance of [[grill-does-not-validate-real-use]] — the suite

columns (KLayout+ref 6.7/8, office at ceiling, R5 6/6) all describe ISOLATED tasks

with a reference in hand, and none of them predict the integrated pipeline.

coder-agentic-default-driver (coder-agentic-default-driver.md)

coder-agentic is the default driver; fable-fusion is the vision fallback

Decided 2026-08-01 after a 3-repeat economy sweep of five candidates plus a

capability profile. coder-agentic = Qwen3-Coder-Next 80B-A3B at UD-IQ2_XXS

(2-bit), 512 experts / 10 per token, arch Qwen3NextForCausalLM, -c 262144

q4_0 KV, 21.71 GiB.

Why it wins on economy — not close

Mean of 3 full runs (33 tasks: 23 coding + 10 agentic), metric_sweep.py:

| model | mean | spread | stable fail | flaky | r5 steps | r5 tok | code tok | wall |

|---|---|---|---|---|---|---|---|---|

| coder-agentic | 31.0 | 0 | expr_eval | 2 | 69 | 4195 | 5121 | 185s |

| fable-fusion | 31.7 | 2 | NONE | 3 | 75 | 6296 | 24742 | 1127s |

| vision-coder | 31.3 | 1 | expr_eval | 1 | 84 | 7083 | 31828 | 1261s |

| bonsai | 30.0 | 0 | expr_eval, selective_rename | 3 | 73 | 7242 | 63863 | 2585s |

| kimi-distill | 30.0 | 2 | expr_eval, wrong_test_triage | 3 | 79 | 10589 | 85351 | 1241s |

| pocket-35b | 29.3 | 3 | expr_eval | 8 | 75 | 7594 | 17053 | 536s |

4.8x fewer coding tokens than fable-fusion, 16.7x fewer than kimi-distill, the

fewest agentic steps and tokens of any model, and 6x faster wall. Its 0.7 deficit

in mean score is inside this box's noise — see

[[single-run-scores-are-unreliable]]; fable-fusion itself spans 31-33.

The most aggressively quantized model on the box is also the most economical.

Third result this week pointing the same way: REAP Q6 scored BELOW Q4, bonsai at

1.71 bpw kept perfect vision. Bit-depth is not where capability lives here.

And it retrieves where the 27Bs cannot

3/3 needle at 237,674 tokens (91% of its window). fable-fusion, vision-coder

and bonsai all score 0/3 at 222k. See

[[qwen36-27b-deep-retrieval-fails]] — the failure tracks the qwen35 attention

design, not long context generally.

Why fable-fusion is still needed

coder-agentic is text-only, permanently — Qwen3NextForCausalLM, no

vision_config, no mmproj in any repo. Nothing to transplant. fable-fusion

covers vision at a measured 21/21, plus 59/59 and 66/66 knowledge, and is the

only model of the five with no task it reliably fails.

Workflow: coder-agentic by default, switch to fable-fusion when an image is

involved. Both live on nvme (~20s swap each way) — see [[nvme-working-set]].

This works for session-shaped work; if you interleave vision turn-by-turn, the

reload cost makes fable-fusion-alone the better answer.

Caveats on the pick

• expr_eval is a stable failure (all 3 repeats). Lineage-wide though —

vision-coder, kimi-distill and pocket-35b all fail it, and fable-fusion misses

it in 2 of 3.

• Untested: vision (N/A), knowledge, and its -x2-style concurrency. Only the

economy + needle axes were measured.

• The sibling entries qwen3-coder-next-80b-160k/-200k share the repo but load

the larger UD-IQ2_M from ssd; they were NOT swept.

CORRECTED 2026-08-09 by [[coder-agentic-regrilled]]

Two claims on this page do not survive the grill_round*.py protocol:

• "spread 0" was measured on ONE suite. Across 9 suites x 2 repeats it is 8,

the joint-highest in the table. Office alone swings 4/5 -> 0/3.

• "6x faster wall" conflates token economy with speed. It is still the

leanest model measured (median 270 coding tok), but it runs at 70 tok/s,

and the same 18-suite set takes 9 min against qwen3-coder's 7.

Still true and re-confirmed: text-only permanently, 0 cap-hits, and it is now

also the joint-best non-fable KLayout model (13/16 with an API reference).

It was NOT re-promoted; qwen3-coder holds the driver slot.