gemma-4-26b-a4b-moe

gemma-4-26B-A4B-it (served as gemma-moe) — TRIED AND REJECTED 2026-07-31

unsloth/gemma-4-26B-A4B-it-GGUF:UD-Q5_K_XL (19.76GiB) + mmproj-BF16, -c 262144

q8_0 KV, --tensor-split 46,54, port 9109. Entry + 21GB deleted same day.

The pitch: the box's only Google entry is coder-accurate, the DENSE gemma-4-31B

at 15-17 t/s. This is the MoE sibling (128 experts, ~4B active) — same lineage, same

gemma4 arch, and it fits UD-Q5 at the FULL 262144 vs coder-accurate's 176k ceiling.

Not a gap fill; a candidate upgrade for that slot.

Why rejected

1. Reproducible runaways. 2 of the first 6 round-1 tasks consumed the entire

16000-token cap. Both reproduced across two independent runs to within 0.6s:

| task | run A | run B |

|---|---|---|

| median_two_sorted | 251.1s / 16000 tok | 250.5s / 16000 tok |

| lru_ttl | — | 250.9s / 16000 tok |

That is a deterministic loop attractor, not sampler noise — the opposite of

the variance described in [[round5-is-a-sample-not-a-measurement]]. Same failure

shape that retired gpt-oss-20b and kimi-linear.

2. Ceiling below the incumbent. The 23-task suite is r1(8)+r2(5)+r3(5)+r4(5).

Two failures cap it at 21/23; coder-accurate holds 22/23. It could not

win the slot it was meant to take, regardless of the remaining tasks.

3. Crashed under sustained load. [WARN] group: running gemma-moe exited:

upstream exited unexpectedly mid-grill. Ruled out as self-inflicted: llama-swap

itself never restarted (NRestarts=0) and sync-models.sh has no service

interaction. Likely chain: the loop attractor drives 16k-token generations ->

deep KV occupancy -> the thin margin blows. **The runaways probably CAUSE the

crash; they are not independent findings.**

The lesson worth keeping: per-token speed is not speed

I promoted this model on "63.7 t/s vs coder-accurate's 17 t/s = 4x faster." That

was per token, and it inverted once verbosity was counted. Head-to-head, same

tasks, same harness, same temp 0.2:

| task | coder-accurate | gemma-moe |

|---|---|---|

| median_two_sorted | PASS 135.6s / 2329 tok | FAIL 250.5s / 16000 tok |

| expr_eval | FAIL 271.6s / 4618 tok | PASS 204.1s / 13183 tok |

| damerau_levenshtein | PASS 85.0s / 1468 tok | PASS 109.6s / 7262 tok |

| articulation_points | PASS 116.9s / 2013 tok | PASS 107.1s / 7118 tok |

coder-accurate round-1 mean is 2352 tok/task; gemma-moe ran ~5-7x that.

Cumulative through 5 tasks: coder-accurate 4/5 in 883.9s, gemma-moe 3/5 in 922.2s

— behind on score AND wall-clock while running 3.8x faster per token.

Always divide throughput by tokens-per-answer before claiming a speed win.

The harness records out_tok per task, so the comparison is free — just read the

incumbent's old log. I made this exact error twice in one session (also on

Ministral-3-14B, dropped unrilled).

Token distribution was strikingly bimodal — 16000, 13183, 7262, 7118, 16000,

2293 — no middle. Tasks either resolve efficiently or spiral to the cap.

Harness gotcha: a dead upstream fakes a 0/5 round

When the child died, the grill kept running and logged 0/5 passed for rounds 2,

3 and 4 — every task a 502 Bad Gateway, none of it a measurement. **A round of

all-zeros with no per-task timings is an infrastructure failure, not a score.**

Check journalctl -u llama-server for upstream exited unexpectedly before

believing any suspiciously bad round, and delete the log so it cannot pollute a

later ranking. (Service unit is llama-server.service, NOT llama-swap.)

Kept from the attempt

The KV discovery that made UD-Q5 at full context possible in the first place —

only 5 of 30 layers are full-attention (1-in-6; rest sliding-window at 1024),

so KV is 21.2 KiB/tok = 5.31GiB at 262144, not the ~17GiB naive layer math gives.

That generalized into [[kv-sizing-full-attention-layers]]. See also

[[vram-spend-priority]] and the coder-accurate entry's measured ladder.

SUPERSEDED 2026-08-10 — see [[gemma-4-26b-awq-vllm]]

The AWQ build on vLLM does not reproduce ANY of the three rejection grounds.

median_two_sorted and lru_ttl, the two deterministic attractors, pass in 808

and 639 tokens. Coding is 44/46, tied best on the box. 1 cap-hit in 24 runs.

The reasoning above is where the error is, not the measurements. "Reproduces

to within 0.6s, therefore a loop attractor in the weights, not sampler noise" —

determinism only shows the cause is deterministic. A chat-template or sampler

default is equally reproducible. The per-token-speed lesson on this page still

stands and is still worth keeping; the attractor conclusion does not.