glm-4-7-flash-reap-23b

GLM-4.7-Flash-REAP-23B-A3B — TRIED AND REJECTED 2026-07-31

unsloth/GLM-4.7-Flash-REAP-23B-A3B-GGUF, three quants tested, all deleted (55GB).

Cerebras REAP-pruned glm-flash: 16 of 64 routed experts removed. Config diff vs

zai-org/GLM-4.7-Flash is exactly one field — n_routed_experts 64 -> 48. Everything

else identical (glm4_moe_lite, 47 layers, experts_per_tok 4, n_shared_experts 1,

kv_lora_rank 512, ctx 202752). NOT the [[model-triage-checklist]] check-2 failure

mode: expert pruning keeps every layer and all attention; only capacity is removed.

The 2x2 (temp 0.2, same binary 10133/ff067f76d, identical sampler + ctx)

| model | experts | quant | coding | R5 | redundant | R5 steps | R5 tok | GiB | runaways |

|---|---|---|---|---|---|---|---|---|---|

| glm-flash | 64 | Q4 | 21/23 | 6/6 | 0 | 30 | 4737 | 16.32 | 2 |

| glm-reap | 48 | Q6 | 20/23 | 6/6 | 0 | 36 | 5752 | 18.84 | 0 |

| glm-reap-q4 | 48 | Q4 | 21/23 | 6/6 | 0 | 36 | 5890 | 13.27 | 1 |

DECISION: kept glm-flash. The challenger matched on score but was 20% worse on

R5 steps / 24% on R5 tokens — the exact property that slot exists for. And a swap would

only have bought 3GB of VRAM that nothing needs (glm-flash already runs with ~3.8-4.0GB

free per card). No unmet need = no swap.

Finding 1: expert pruning is ~FREE

At matched Q4, cutting 25% of experts cost zero coding answers — 21/23 both,

identical round breakdowns (7/8, 4/5, 5/5, 5/5), same text_justify miss. Makes sense

mechanically: only 4 routed experts fire per token whether the pool is 48 or 64, and

REAP selects victims by router-weighted activation, i.e. the least-used ones. **Pool

size is capacity, not per-token compute** — so pruning cannot slow decode either.

Finding 2: MORE BITS DID NOT HELP — Q6 scored BELOW Q4

At identical expert count, Q6 got 20/23 and Q4 got 21/23, and Q6 decoded 1.3x

SLOWER (63 vs 82 t/s — Q6 weights are 1.42x larger and decode is bandwidth-bound).

Second independent result on this box pointing the same way; the first was the

agent-mini Q4-vs-Q8 grill (no reliable gain, 38% slower — see the entry comment).

The "bigger quants" half of [[vram-spend-priority]] is not supported for WEIGHTS.

The KV-precision half is untouched by this.

Caveat: the Q6-vs-Q4 delta is ONE task (wildcard_match) — planned 3x repeats were not

run before removal, so treat as "no evidence higher weight-quant helps", not proof it hurts.

Finding 3: the agentic regression that WAS real

test_driven_loop took 10 steps (Q6) and 11 (Q4) vs glm-flash's 4 — reproduced

at BOTH bit-depths, so quant is excluded and it IS a pruning effect. Confined to that

one task; the other five matched or beat the parent. This is what decided the slot.

Finding 4: I over-read the runaway data (correction)

I claimed "pruning removes runaways, 2 for 2" after seeing both REAP variants avoid

glm-flash's expr_eval/articulation_points spirals. Wrong — Q4 then spiralled on

text_justify (16000 tok) where neither other model did. Tally: glm-flash 2, Q6 0,

Q4 1. Runaway propensity is task-specific and moves with both variables. Two tasks

is not a causal claim.

Q8_0: rejected on MARGIN, not behaviour

Q8_0 (22.78GiB) at 202752 passed every functional check — 5/5 long generations all

finish=stop, deep-context prompt fine, no crash — but left only 664MiB / 853MiB

free, inside the coder-accurate ladder's "unreliable, dies under VRAM pressure" zone.

Surviving an isolated test on an idle box is not the bar; llama-swap swaps under

pressure. Held the line rather than rationalising a survived test (the mistake made

hours earlier with [[gemma-4-26b-a4b-moe]], which passed one generation then died

mid-grill). Predicted 2.4GB free, actual 664MiB — see [[kv-sizing-full-attention-layers]]

trap 3 on why the glm-flash-anchored buffer estimate was ~1GB light for Q8 weights.

glm-flash's real score is 21/23, not 20/23

The memory index said 20/23; bench/results-glm-flash-20260727-151214.log shows

7/8 + 4/5 + 5/5 + 5/5 = 21/23, plus R5 6/6 with 0 redundant on every task at

447-1079 tok/task and 70-90 t/s. That R5 profile is what "best agentic efficiency on

the box" means, and it is the bar any GLM-slot challenger must clear.

Method note worth reusing

The first run confounded TWO variables (48-vs-64 experts AND Q6-vs-Q4) and could prove

nothing. One extra 13GB download (Q4 at 48 experts) filled the third cell of a 2x2 and

isolated both main effects. **When a candidate differs from its baseline in two ways,

find the third cell before drawing conclusions** — it is usually one cheap download.

Also: GRILL_URL lets the harness hit a standalone llama-server directly, so a trial

model can be grilled without touching llama-swap.yaml or needing a service restart.