← all models

tess-4-27b

Retired  no longer in llama-swap.yaml — entry and/or weights removed after this verdict.

Grill run history

RunScoretok/stepLog
20260727-13403422/23241.3results-tess-4-27b-20260727-134034.log
20260725-23451722/23237.6results-tess-4-27b-20260725-234517.log

Memory note

tess-4-27b — RETIRED 2026-07-27, NO LONGER SERVABLE

Removed from the box 2026-07-27 ([[2026-07-27-model-cleanup]]): entry

deleted from llama-swap.yaml, 27GB Q6_K + MTP head wiped from the HF cache.

Rationale: a genuine grill-round-5 miss on top of already being one below

fable-fusion on the base 23-task grill. Everything below is history.

Original entry — migtissera/Tess-4-27B (Q6_K, 262144, port 9114)

Added 2026-07-26. Agentic/reasoning fine tune of Qwen/Qwen3.6-27B by Migel

Tissera, Apache-2.0 — post-trained on 64K-token agentic traces with a reasoning

style approximated from Fable-5 by an Opus-4.8 / GPT-5.5 / GLM-5.2 teacher

ensemble. Same qwen35 hybrid SSM/attention arch as [[fable-711-27b]] and

vision-coder: 64 layers but full_attention_interval 4, so only 16 carry KV

(n_kv_head 4, head_dim 256) → ~18 KiB/tok at q4_0, which is why 256k is

affordable on 32GB.

GRILL 22/23 (bench/results-tess-4-27b-20260725-234517.log, temp 0.2):

r1 7/8, r2 5/5, r3 5/5, r4 5/5. **Ties vision-coder; one below

[[fable-711-27b]]'s 23/23, which keeps the best-all-rounder crown.**

  • Single miss: expr_eval — the unary-minus-vs-power precedence case.

Burned 10954 tok / 762s and still emitted ValueError: Invalid number: *.

fable passes it in 8337 tok; [[gpt-oss-20b]] clears it ~1/3 of the time.

This one task is the discriminator at the top of this box's leaderboard —

it is what separates 23/23 from 22/23.

  • Passes text_justify (the column-width off-by-one vision-coder fails).
  • lru_ttl PASSED but hit the 16000-token cap (1123s) — a fragile pass;

re-check it first if that task ever regresses.

  • Rounds 2-4 were tight (2.4k-5.4k tok algorithmic; 66-861 tok tool-call/format;

258-859 tok / 0-3 steps agentic), so over-deliberation is confined to hard

round-1 algorithmics, NOT a general trait.

MTP RESOLVED 2026-07-26 — the suspicion was right, and it is now ON.

Controlled A/B (same -c 131072, --tensor-split 55,45, same sampler; only the

two spec flags differ; 3 × 900-token generations each):

| | tok/s | draft acceptance |

|---|---|---|

| MTP off | 14.31 / 14.31 / 14.33 | — |

| MTP on | 31.06 / 31.44 / 31.93 | 0.82-0.86, mean len ~2.7 |

2.2×, landing exactly on fable-fusion's 32-35. Quality is unaffected —

draft-mtp verifies every drafted token against the main model, so this is not a

speed-for-quality trade; rounds 3+4 re-run 10/10 with it on.

  • The flag is -md <head> --spec-type draft-mtp, NOT --mtp — --mtp does

not exist in b9870. Head passed by absolute snapshot path (-hff already pins

the main GGUF); re-point on a re-pull. Q8_0 head (3.16GB) is the one in use —

the Q4_K_M head was never needed.

  • It costs half the window. 262144 + head OOMs both ways (CUDA1 at 48,52;

CUDA0 at 62,38 — that attempt *segfaulted* after graph_reserve failed).

196608 loads at 55,45 and runs 30.8 t/s but leaves <700MB free per card,

and only after llama.cpp retries without pipeline parallelism — too thin to

serve. 131072 at 55,45 is the setting: ~29.5GB used, real headroom.

  • Alias renamed tess-4-27b-256k → tess-4-27b-128k. To restore a 256k

tess, drop the two spec flags and put back -c 262144 --tensor-split 48,52 —

that is the exact config the 22/23 grill ran on.

  • Trade accepted because [[fable-711-27b]] already covers the 256k + ~32 t/s

slot at 23/23; a 14 t/s tess had no niche, a 31 t/s one is a usable second

opinion. Original no-MTP full run was ~90 min / 77.6k output tokens.

Quant choice — Q6_K, and the reasoning generalizes to any 27B on this box:

  • Q8_0 (26.63GiB) + mmproj (0.87) = 27.5GiB → ~3.5GiB left → ~110k ctx after

compute buffers, under the 128k floor. REJECTED.

  • Q4_K_M (15.41GiB) fits trivially but wastes ~5GiB and duplicates the tier

fable already occupies at this exact size/arch.

  • Official migtissera/Tess-4-27B-GGUF over bartowski's mirror: wins downloads

(26k vs 12.5k) and likes (31 vs 10), and its Q6_K is 1.3GiB SMALLER

(20.57 vs 21.85) — that 1.3GiB is worth more as KV here than bartowski's

imatrix edge is at Q6. Official is also the only one shipping matching mtp-

heads.

--tensor-split 48,52 is MEASURED. At 50,50 it loads at 262144 but leaves

CUDA0 (4060 Ti) only 691MiB free vs CUDA1's 1364MiB — thinner than fable's

verified 1.1/1.4GB and risky for a large-image ViT encode. 48,52 evens it to

993MiB / 1060MiB. Total free is conserved (~2.05GB); the split only slides

which card is tightest. See [[gpu-device-ordering]].

Two things that went RIGHT here, unlike its siblings:

  • mmproj auto-discovers from the HF cache under LLAMA_ARG_OFFLINE=1 —

exactly ONE projector in the repo, so no absolute snapshot path to rot on a

re-pull (fable and bonsai both need pinned paths).

  • -hff is still passed explicitly: the repo also holds

mtp-Tess-4-27B-Q4_K_M.gguf, so -hf repo:Q4_K_M *would* be ambiguous. Q6_K

happens not to collide, but naming the file keeps that from being load-bearing.

Template: same guard landmine. Embedded template is byte-identical to base

Qwen3.6-27B and raise_exceptions on any non-first system message → 400

"Unable to generate parser" the moment a client sends tools. Patched copy at

chat-templates/tess-4-27b-256k.jinja; it diffs against the proven

chat-templates/fable-711-256k.jinja on exactly the guard line. See

[[jinja-system-guard-tool-parser]].

Grill-without-sudo trick used here: llama-swap has no reload endpoint and a

restart needs sudo, so the model was hand-started on 9114 with the yaml's exact

cmd: and grilled via run_grill.sh tess-4-27b http://127.0.0.1:9114/v1/chat/completions

(the URL is positional). Also note .claude/sync-models.sh VISION_IDS is

hand-maintained — tess-4-27b had to be added by hand or models.json would call

it text-only ([[sync-models-hook]]).