Kept / trial active in llama-swap.yaml · aliases: pocket-35b-a3b-qwen3.5-256k
| Run | Score | tok/step | R5 | Log |
|---|---|---|---|---|
| 20260728-082609 | 22/23 | 125.1 | — | results-pocket-35b-20260728-082609.log |
FINAL-Bench/POCKET-35B-GGUF:Q4_K_M, 19.71 GiB. Apache 2.0. The base_model
field says FINAL-Bench/Darwin-36B-Opus, but **the card's own badge says the
real base is Qwen3.5-35B-A3B** — treat the Darwin/POCKET naming as branding.
Real sparse MoE (256 experts, 8/tok, ~3B active), arch qwen3_5_moe, 40 layers,
head_dim 256, kv_heads 2, vocab 248320, 262144 ctx, no MTP head. Text-only as
served. Port 9115, full -c 262144 with q8_0 KV at split 45,55 → 1742/3820 MiB
free.
| | grill | R5 | loops | redund% | tok/step | decode |
|---|---|---|---|---|---|---|
| pocket-35b | 22/23 | 6/6 | 0 | 0.0 | 125.1 | 102.1 (peak 107.9) |
| [[kimi-distill]] | 22/23 | 6/6 | 0 | 0.0 | 160.1 | 102.5 |
Only miss: expr_eval, the lineage-wide hard case kimi-distill also fails.
Round 5 a clean sweep, zero redundant calls. **Leaner per step than
kimi-distill** (125.1 vs 160.1) at the same speed.
Peer, not an upgrade — and per caveat 1 a single 22 vs 22 is inside the flake
band. It is arguably redundant with kimi-distill the way
[[kat-coder-v2.5-dev]] was, except this one actually TIED rather than scoring
below, so the redundancy call is genuinely a coin-flip rather than obvious.
Everything but the base generation was held constant (quant class, 262144 ctx,
q8_0 KV, split, sampler, grill settings). Identical on every graded axis. No
evidence of a generational gain at this size on this suite.
Recommended SKIP before running, on two grounds:
1. vendor pitch is CPU/on-device/phone ("runs with no GPU") — irrelevant on a
32GB-VRAM box. This ground was FINE; it just says the marketing isn't for us.
2. "base is a generation behind what's already served" — WRONG, and it
was the ground carrying the actual recommendation.
Deployment-target mismatch says nothing about capability. Base-generation age
predicts nothing. Only a grill measures capability. Corrected in
[[model-triage-checklist]] check 8 and MODEL-RANKING.md caveat 11. The broader
lesson: triage checks are only sound when they bear on whether the model can
RUN or whether the benchmark can MEASURE it — not on how modern or well-marketed
it looks.
Its embedded template carries raise_exception('System message must be at the
beginning.') at line 85 ([[jinja-system-guard-tool-parser]]). Plain chat 200,
single-turn tools 200, multi-turn system+tools 400. Only the multi-turn
probe catches it; had it not been caught, rounds 4/5 would have failed as
REQ-ERR and produced a fabricated score.
Its template is BYTE-IDENTICAL to kat-coder's (both the Qwen3.5/3.6-MoE
family template, 7764 chars), so the previously patched copy worked verbatim —
renamed chat-templates/qwen35moe-nonfirst-system-256k.jinja to reflect that it
is a family-wide fix. --chat-template-file is load-bearing.
NOT affected by [[llama-cpp-peg-native-tool-parser-500]]: prose-with-tools
returns 200 with content "42", unlike the Llama-3.3 template.
pocket-35b-x2 (port 9118, -c 262144 total => 131072/slot): grill 21/23,
R5 6/6 single-stream, **concurrent R5 6/6 + 6/6, 0 loops, 0.0% redundant on
both streams**. 103.3 t/s, tok/step 123.4.
Across four independent round-5 runs of these weights (solo, x2 single-stream,
x2 concurrent A, x2 concurrent B): **6/6 every time, 24/24 tasks, zero loops,
zero redundant calls** — including interrupt_replan (which [[north-mini-code]]
fails ~5/6) and search_navigate (where coder looped).
That makes it the best-evidenced R5 on the box and the counterexample to
[[round5-is-a-sample-not-a-measurement]]: repetition is not only for exposing
flakes, it also *confirms* stability. R5 variance is a property of the model,
not harness noise. Recorded as MODEL-RANKING caveat 16.
pocket-35b-x2 is now the concurrent agentic pick — same clean behaviour as
fable-fusion-x2 at 3x the speed (103.3 vs 32.9 t/s); reach for fable-fusion-x2
only when vision is needed. Its 21/23 (vs solo 22/23, missing expr_eval and
dijkstra) is inside the ±1-2 flake band — here the SHORT suite is the noisy
one, not round 5.
"pocket-35b":
aliases: [pocket-35b-a3b-qwen3.5-256k]
# FINAL-Bench/POCKET-35B-GGUF:Q4_K_M (19.71GiB) — TRIAL 2026-07-28.
# Despite the "Darwin-36B-Opus" base_model field, the card's own badge says the
# real base is **Qwen3.5-35B-A3B**. Real sparse MoE (256 experts, 8/tok, 3B
# active), arch qwen3_5_moe — same family llama.cpp already runs for the
# Qwen3.6 entries. 40 layers, head_dim 256, kv_heads 2, vocab 248320, 262144 ctx.
# No MTP draft head. TEXT-only here (no --mmproj passed).
#
# WHY IT WAS TRIED, despite triage saying skip: the box has never grilled a
# Qwen3.**5**-35B-A3B, only 3.6 derivatives (chat / uncensored / kimi-distill).
# So this measures what the 3.5 -> 3.6 GENERATION jump actually bought, using
# kimi-distill's exact serving config as the paired comparison. The vendor pitch
# (CPU / on-device / phone, "no GPU needed") is irrelevant on this box and is NOT
# what is being tested — see [[model-triage-checklist]] check 8.
#
# SAMPLER/CONFIG deliberately identical to the kimi-distill entry (temp 0.6 /
# top-p 0.95 / top-k 20 / min-p 0, q8_0 KV, 256k, split 45,55) so the only real
# variable is the model generation. Grill overrides temp to 0.2 for parity.
#
# TEMPLATE: HAS the non-first-system guard (line 85), confirmed live 2026-07-28.
# Single-turn tools returns 200; only a MULTI-TURN system+tools request 400s with
# "Unable to generate parser for this template" — so a single-turn smoke test does
# NOT catch it. Its embedded template is BYTE-IDENTICAL to kat-coder's (both are
# the Qwen3.5/3.6-MoE family template, 7764 chars), so the same patched copy works
# verbatim — hence the family-neutral filename rather than a per-model one.
# NOT affected by the peg-native parser bug ([[llama-cpp-peg-native-tool-parser-500]]):
# prose-with-tools returns 200 with content "42", unlike Llama-3.3.
# --chat-template-file is LOAD-BEARING; dropping it reintroduces the 400.
#
# REASONING MODEL — budget max_tokens generously. CoT goes to
# `reasoning_content`, the answer to `content`. Measured: answering "ready"
# cost 274 completion tokens / 1025 chars of reasoning; at max_tokens=40 it
# hits the cap mid-think and returns an EMPTY `content` with no error. Clients
# that read only `content` (and short-max_tokens health checks) will look
# broken when they are not.
# GRILL 2026-07-28 at temp 0.2: 22/23 (only expr_eval), R5 6/6, 0 loops,
# 0.0% redundant, 102.1 t/s, tok/step 125.1 — ties kimi-distill on every
# graded axis from a generation-older base. See MODEL-RANKING.md caveat 11.
# *** -sm tensor (was `--tensor-split 48,52`). DO NOT RESTORE --tensor-split: passing it
# with -sm tensor does not start -- llama_params_fit is unimplemented for
# SPLIT_MODE_TENSOR and the load dies on a bogus ~20 GB "cudaMalloc failed"
# that is NOT an OOM. Tensor mode splits itself, evenly, with nothing to tune.
# Layer split runs the cards SEQUENTIALLY so the faster 5060 Ti idles waiting;
# tensor runs them concurrently. auto-memory/sm-tensor-rejects-tensor-split.md
#
# GRILLED 2026-09-02 BOTH ARMS SAME-DAY (bench/grill-smt-pocket-35b-*,
# r1-r4 n=2 + R5 n=3 per arm, same port + same cmd so -sm is the ONLY variable):
# coding 21/23 20/23 tensor | 21/23 21/23 layer
# R5 6/6 6/6 6/6 tensor | 6/6 5/6 5/6 layer
# R5 LOOPS 0/6 IN EVERY REP OF BOTH ARMS <- the signal that matters
# decode 110.4 vs 93.8 tok/s in-suite over 46 tasks/arm = +17.7%
# Differences are within this box's known R5/coding sampling spread.
cmd: |
/usr/bin/env
CUDA_DEVICE_ORDER=PCI_BUS_ID
${llama_bin}
-hf FINAL-Bench/POCKET-35B-GGUF
-hff POCKET-35B-Q4_K_M.gguf
--alias pocket-35b
--jinja --chat-template-file chat-templates/qwen35moe-nonfirst-system-256k.jinja
-ngl 99 -c 262144 -fa on
-sm tensor
-b 2048 -ub 512
--temp 0.6 --top-p 0.95 --top-k 20 --min-p 0.0
--cache-type-k q8_0 --cache-type-v q8_0
--host 127.0.0.1 --port 9115 --parallel 1
proxy: http://127.0.0.1:9115