gemma-4-26b-awq-vllm

Trialled 2026-08-10, cyankiwi/gemma-4-26B-A4B-it-AWQ-4bit, 16.0 GiB,

Gemma4ForConditionalGeneration. Round-1 probe (vllm/probe-gemma-awq.sh) then

the full 12-suite set x 2 repeats in 13 minutes, 0 failed suites — the

fastest full vision grill measured here.

| suite | combined | field position |

|---|---|---|

| coding R1-R4 | 44/46 | TIED BEST (both fables) |

| R5 agentic | 11/12 | 0/6 looped BOTH repeats, 0 redundant calls |

| klayout from memory | 0/16 | field norm |

| klayout + API ref | 12/16 | mid, spread 0 |

| klvision | 10/12 | tied best, spread 0 |

| closed loop, no ref | 0/8 | genuine (post-fix prompt) |

| closed loop + ref | 4/8 | 0/4 then 4/4 — spread 4 |

| office / office+ref | 18/18 / 12/18 | joint best unassisted (see below) |

| cap-hits | 1 | 2nd best |

| throughput | 93 tok/s | |

The July rejection is overturned on every stated ground

[[gemma-4-26b-a4b-moe]] rejected the GGUF on 2026-07-31 for (1) reproducible

16k runaways, (2) a 21/23 ceiling, (3) crashing under load. Measured here:

median_two_sorted 251.1s / 16000 tok -> 8.5s / 808 tok PASS

lru_ttl 250.9s / 16000 tok -> 6.8s / 639 tok PASS

Coding is 44/46, not 21/23. Zero crashes. 1 cap-hit in 24 suite runs.

THE INFERENCE ERROR WORTH KEEPING. That session saw two tasks burn the full

cap reproducing to within 0.6s and concluded "a deterministic loop attractor,

not sampler noise — the opposite of variance". Determinism only proves the CAUSE

is deterministic; a chat-template or sampler-default bug in the serving stack is

exactly as reproducible as a weights property. Engine and quant changed together

here, so which one fixed it is UNKNOWN — but the weights are no longer a

defensible explanation. Compare [[single-run-scores-are-unreliable]]: that file

warns against trusting one run; this is the opposite failure — trusting

reproducibility as proof of location.

Lineage diversity finally works

[[ernie-45-vl-28b-thinking]] was rejected as a non-Qwen probe because it scored

0/8 even WITH the KLayout API reference, worth +4..+13 to every Qwen model.

gemma converts it: 0/16 -> 12/16, spread 0. So that failure was ERNIE's, not a

property of non-Qwen models, and this box now has a real lineage-diverse option.

Relevant to [[qwen36-27b-deep-retrieval-fails]], the architectural weakness the

Qwen entries share.

Office was NOT a hole — it was a harness defect

First measured 0/18 on both arms. Cause: gemma writes

if __name__ == "__main__": main() and the grills exec'd with ns = {}, so

__name__ resolved to "builtins" and the program never ran. Re-measured under

the fix: 18/18 unassisted, spread 0 — joint-best with glm-flash-awq, and

gemma is the only model that manages it while ALSO having vision. See

[[exec-main-guard-defect]]. The assisted arm is 12/18, held back by a malformed

snippet in the reference text (now fixed), not by the model.

The real gap is KLAYOUT: 0/16 from memory, 12/16 with the reference, 0/8 on

the closed loop. Point KLayout work at fable-fusion.

Long context VERIFIED — 3/3 at 247,592 real tokens

Needle test 2026-08-10 (bench/needle.py, depths 0.1/0.5/0.9):

54,696 real tokens 3/3 ~15s per query

204,960 real tokens 3/3 ~110s

247,592 real tokens 3/3 ~150s

Served at its NATIVE 262144, not the 131072 first configured. KV is nearly free

here — only 5 of 30 layers are full_attention (rest sliding-window at 1024),

so fp8 KV is 20 KiB/token = 2.50 GiB/GPU at the full 262k. Measured at 131072 the

server reported 4.16 GiB KV free (597,899 tokens of pool), i.e. it was using

under a quarter of its headroom. See [[kv-sizing-full-attention-layers]].

This beats coder-agentic's 237k, which was the box's only MEASURED deep

retrieval record — so the long-context slot now belongs to the driver, and

coder-agentic was demoted to ssk500 the same day ([[nvme-working-set]]).

The Qwen3.6 retrieval collapse is lineage-specific.

[[qwen36-27b-deep-retrieval-fails]] records those models 3/3 at 32k and 0/3 at

222k. gemma is clean at 248k, so that finding constrains Qwen3.6-27B

architecture, NOT long context generally.

Harness trap: needle.py's TARGET_TOKENS is NOT the prompt size — the filler

tokenises ~1.7x denser (label 120000 -> 204,960 actual). A label of 250000 lands

at ~427k and is REJECTED on context length, which reads as a model failure and is

not one.

Serving notes

• --tool-call-parser gemma4 — hermes returns tool_calls: null with no

error, which would silently null R4 and R5. Enumerate valid names with

ToolParserManager.list_registered() (44 of them; the manager is at

vllm.tool_parsers, NOT vllm.entrypoints.openai.tool_parsers).

• --reasoning-parser gemma4 IS REQUIRED — easy to miss, and I missed it.

gemma4 emits no <think> block, but it wraps reasoning in CHANNEL markers;

tokenizer_config.json states the format outright:

(<|channel>thought\n(?P<thinking>.*?)<channel|>)?(?P<tool_calls>...).

Without the parser that text lands in content VERBATIM. Measured

2026-08-10: the turn after a tool_result returned

"<|channel>thought\n<channel|>The task has been completed successfully."

Plain turns look clean, so it only surfaces in agentic use — a grill that

extracts fenced code blocks tolerates the prefix and scores normally. Verified

fixed with vllm/test-claude-code-path.py (5/5 including a leak check).

• 16.0 GiB = 8 GB/GPU at TP=2, roomy.

PROMOTED — default driver across all 6 agent harnesses as of 2026-08-21

(claude-local/opencode/kimi-code/omp were already default; pi + prime added;

claude-local doc reconciled). It is the fastest top-scoring coder here (44/46

at 93 tok/s vs the fables' 28-35) with tied-best vision (10/12) and joint-best

office (18/18 unassisted, after the [[exec-main-guard-defect]] fix), plus 248k

VERIFIED deep-retrieval — the deciding factor over fable-fusion, whose perfect

169/169 grill is a trap for real codebase work (needle 0/3 @222k,

[[qwen36-27b-deep-retrieval-fails]]). The ONE real gap is KLayout (0/16 from

memory, 12/16 +ref, 0/8 closed loop) — point that at fable-fusion. loop+ref

0/4 then 4/4 means the loop capability is present but unreliable — see

[[single-run-scores-are-unreliable]].