Kept / trial active in llama-swap.yaml · aliases: qwen3.8-27b-turbo-fcf-735-882-256k
| Run | Score | tok/step | R5 | Log |
|---|---|---|---|---|
| No results-*.log found for this id/alias. | ||||
fable-turbo (alias qwen3.8-27b-turbo-fcf-735-882-256k), port 9196.
DavidAU/Qwen3.8-27B-TURBO-Fable-Cold-Fusion-735-882-Heretic-Uncensored-NEO-CODER-MAX-MTP-GGUF,
quant MTP-Q4_K_M 17.23 GiB + mmproj-BF16 on /mnt/models (cold tier, it is a trial).
Entry is a deliberate clone of [[fable-711-27b]]'s serving config (262144, -fa on,
-sm tensor, --spec-type draft-mtp, temp 0.6, KV q4_0) so weights were the only variable.
The README (line 129) says it outright:
> it contains BOTH "Qwen3.6-27B-Fable-Fusion-711-Uncensored-Heretic" (DARK ROAST
> VERSION) and "Qwen3.8-27B-Cold-Fusion-GAIN-V1.1" as part of it's critical/core "DNA"
So the A/B I ran was **not "successor vs incumbent", it was "a merge containing the
incumbent vs the incumbent"** — and parity on 9 of 11 arms is the STRUCTURALLY EXPECTED
result, not a coincidence. 735-882 in the name is just his own arc scores
(arc/c 0.735, arc/e 0.882). The Qwen3.8 ancestry is real
(trohrbaugh/Qwen3.8-27B-heretic-ara off Qwen/Qwen3.8-27B) but names an INGREDIENT.
DavidAU's own table has the Qwen3.8 base LOSING to the Qwen3.6 base on every column
(arc/c .591 vs .647, arc/e .782 vs .803, hswag .746 vs .773, wino .711 vs .742), and he
rates the Qwen3.8 parent "level 1 or 2" against fable-fusion's "level 7-8". The newer
generation is the WEAKER half of this merge.
Dumped all three local GGUFs. Every architectural field is byte-identical, and all
three have exactly 866 tensors:
arch qwen35 | block_count 65 | embedding 5120 | ffn 17408 | head_count 24
head_count_kv 4 | key/value_length 256 | rope 1e7 | ssm.* all equal
full_attention_interval 4 | nextn_predict_layers 1 | context_length 262144
fable-turbo, qwen38 (a real Qwen3.8-27B) and fable-fusion (Qwen3.6-27B) are
indistinguishable this way. **A header dump proves the model SERVES on ${llama_bin};
it proves NOTHING about generation.** The only generation evidence is author-set free
text (general.basename, general.name) plus the chat template. Note fable-fusion's
GGUF carries general.base_model.count = 7 with full provenance URLs and **this one
carries no base_model.* at all** — absent provenance is itself a signal to go read
the card.
bench/grill-fable-turbo.sh (clone of grill-thomson.sh), results in
bench/grill-fable-turbo-20260909-1306/.
| arm | rep1 | rep2 | rep3 | mean | fable-fusion |
|---|---|---|---|---|---|
| 23-task | 23 | 22 | 23 | 22.67 | 22.3 |
| R5 complete | 5/6 | 5/6 | 5/6 | 5/6 | 6/6 |
| R5 looped / redundant | 0 / 0% | 0 / 0% | 0 / 0% | 0 of 18, 0.0% | 0 |
| interrupt_replan | PASS | PASS | PASS | 3/3 | pass |
| klayout unaided | 2/8 | 2/8 | 3/8 | 2.33 | 1.3 |
| klayout +ref | 7/8 | 6/8 | 8/8 | 7.0 | 7.7 |
| klvision | 5/6 | 5/6 | 5/6 | 5.0 (ceiling) | 5.0 |
| closedloop unaided | 4/4 | 4/4 | 2/4 | 3.33 | 2.67 |
| closedloop +ref | 4/4 | 4/4 | 4/4 | 4.0 | 4.0 |
| office (both arms) | 9/9 x2 | 9/9 x2 | 9/9 x2 | 54/54 | 5 of 6 arms |
| realcase gate | PASS | PASS | PASS | 3/3 | 3/3 |
| cap-hits | 0 | 0 | 1 | 1 in ~130 tasks | 0-2 |
VERDICT: KEEP AS TRIAL, DO NOT PROMOTE. Parity is redundancy — we already serve
Qwen3.8-27B three ways (qwen38, qwen38-awq, qwen38-mtp) and half of this model's
DNA is the incumbent it would displace.
The one clean win is office 54/54, and it is VERIFIED not a false pass. Read the
workbooks by hand: real =B2*C2 / =SUM(D2:D5) formulas, and the chart ref D2:D5
correctly EXCLUDES the Grand Total row (it looks like a dropped last row and is not).
It avoids the Reference/titles_from_data off-by-one that THREE models hit
independently in [[klayout-office-model-comparison]] and that fable-fusion still built
into its one perfect office run.
Only cap-hit was grow_shapes in klayout-UNAIDED: 32000 tok / 675.7s, wrong answer too.
Not on the +ref production path.
grill_textturn.py n=8 x 3 reps. **Perfectly localised, not the intermittent leak the
suite was built for:**
| scenario | defective |
|---|---|
| A_no_tools | 23 of 24 (22 leak <tool_call>, 1 FABRICATED output) |
| B_tools_text | 0 of 24 |
| C_post_tool_summary | 0 of 24 |
With no tools available it does not decline — it emits a raw
<tool_call>{"name":"shell",...}</tool_call> as visible content, or once invented a
plausible ls -ltr listing (total 12 / drwxr-xr-x ... old.txt) it never ran.
Fabrication is a WEIGHTS defect no sanitizer can strip.
With tools present it is clean 48/48, and stream parity is clean 48/48. Real agent
turns always carry tools, so pi/Claude Code drive it fine — verified tool_use on the
real :8091 /v1/messages route, not just chat/completions ([[tabbyapi-no-anthropic-messages-endpoint]]).
The narrow exposure is genuine tools=0 traffic, which Claude Code's auto-mode
classifier does send ([[claude-code-local-model-blockers]] blocker 3) — UNTESTED whether
that actually fails the classifier closed.
* THE ENVELOPE REPORT SAID "CLEAN 0/372 turns" WHILE THIS WAS BROKEN. * Exactly the
[[aquila-think-tag-leak-real-use]] trap: one suite passing does not clear a model, and
these tags are tool-call syntax, not the control tokens the envelope counts.
nested_rings is RESOLVED and should not be investigated a fifth time. The
generator draws exactly 3 annuli — [(20000,17000),(14000,11000),(8000,5000)], ground
truth {"rings": 3} — and the render is legible. FOUR models now fail it converging on
the same wrong answers (fable-711-gptq 5, qwen3-vision 4, fable-fusion all 3 batteries,
fable-turbo 5 then 4). It is well-formed but has ZERO discriminating power:
klvision 5/6 is the effective ceiling here, and a 6/6 would be the interesting event.
Closes the open question in [[klayout-office-model-comparison]].
**closedloop is a 2-TASK suite scored out of 4 — one code-stage failure moves it by
HALF.** rep3's 2/4 was a single loop_array:code miss taking its dependent vision stage
with it. That fully explains fable-fusion's documented 2/4 -> 4/4 -> 2/4 swing: those are
one-task differences, not capability swings. **At n=3 this suite cannot rank two close
models** — I read "4.0 vs 2.67" as a real edge after 2 reps and rep3 erased it.
Same class as [[single-run-scores-are-unreliable]].
Do NOT read interrupt_replan 3/3 as beating [[kat-coder-mtp-gguf-trial]]'s 1/14.
That entry exists because kat looked like 0/5 before n=14 found a 7% rate; this box needs
n>=14 for a stochastic failure. fable-turbo's full-ROUND record is genuinely cleaner
at equal sample (0 loops + 0 redundant over 18 scenarios vs kat's 3 loops and one 67.7%
redundancy blowup) but the per-task claim is unsupported at n=3.
Runtime estimate was wrong by 15x — 103 min, not the 18-30 h I projected. I scaled
from grill-thomson.sh by decode speed alone. **Wall time is driven by TOKENS EMITTED, not
tok/s**: this model is strikingly concise (3274 out_tok across all 6 R5 tasks). Estimate
future batteries from a model's verbosity, not its throughput.
acceptance 71-82% — well above the card's 50% fall-back-to-non-MTP threshold, so
MTP-Q4_K_M is the right file. Speed is PARITY with fable-fusion's 44.4 under -sm tensor.
nothing — [[qwen35moe-35b-vram-ceiling]]): 1434 MiB free CUDA0 / 2841 CUDA1. So
MTP-Q5_K_S (+1.98 GiB) is marginal-but-plausible at 262144 and MTP-Q5_K_M (+2.50)
would almost certainly OOM on a large image. Q6_K/Q8_0 leave no context.
chat-templates/qwen3.8-27b-nonfirst-system.jinja — it is the QWEN3.8 template
(reasoning_effort xhigh/medium/low, video, image counting), NOT the Qwen3.6 one, so
fable-711-256k.jinja is the WRONG patch to reuse. Patched copy:
chat-templates/turbo-fcf-735-882-256k.jinja. See [[jinja-system-guard-tool-parser]].
reasoning_effort=xhigh, so CoT is long; budget max_tokens.1. Re-run rounds 1-4 at GRILL_TEMP=0.6 — kat-coder-fast was graded at 0.6 and
fable-turbo at 0.2, so the 22.67-vs-20.33 coding gap is CONFOUNDED by sampler.
2. grill_round5.py interrupt_replan at n=14, to put it on kat's footing.
3. Whether the tools=0 fabrication actually fails Claude Code's auto-mode classifier.
"fable-turbo":
aliases: [qwen3.8-27b-turbo-fcf-735-882-256k]
# TRIAL 2026-09-09 — DavidAU/Qwen3.8-27B-TURBO-Fable-Cold-Fusion-735-882-Heretic-
# Uncensored-NEO-CODER-MAX-MTP-GGUF, quant MTP-Q4_K_M (17.23 GiB) + mmproj-BF16.
# The card bills this as the SUCCESSOR to Qwen3.6-27B-Fable-Fusion-711, i.e. to
# `fable-fusion` — the box's quality pick and the only model that closes the
# KLayout loop unaided. So this entry is a DELIBERATE CLONE of that one: same
# -c, -fa, -sm, --spec-type, sampler and KV types, so the WEIGHTS ARE THE ONLY
# VARIABLE. Do not "improve" any flag here before the A/B is graded — that is
# the method auto-memory/davidau-catalog-verdicts.md prescribes and the reason
# three prior DavidAU trials produced trustworthy rejections.
#
# ARCH VERIFIED FROM THE GGUF HEADER (range request, before downloading):
# general.architecture qwen35 <- runs on the pinned ${llama_bin}, no fork
# block_count 65, context_length 262144, full_attention_interval 4,
# key/value_length 256, head_count_kv 4, nextn_predict_layers 1
# That is STRUCTURALLY IDENTICAL to fable-fusion: only 16 of 65 layers carry a
# KV cache, so 262144 costs 4.00 GiB at q4_0, and the MTP head means
# --spec-type draft-mtp applies. Weights are the SAME 17.23 GiB as
# fable-fusion's file, so the proven 262144 + -sm tensor fit carries over.
#
# *** NOT A BRAINSTORM EXPANSION — this is why the DavidAU prior does not
# condemn it. *** block_count is 65, the same as the base; the three rejected
# DavidAU models (GLM-Grande-42B 67 layers vs base 48, Deckard-40B, gpt-oss
# abliterated) were layer expansions or abliterations that converted bounded
# misses into 16k runaways. The two DavidAU models we KEPT (fable-fusion,
# glm-flash) are un-expanded. This one is un-expanded. Heretic-decensored,
# which is also true of fable-fusion.
#
# QUANT CHOICE: MTP-Q4_K_M, not bigger. The repo ships 22 quants; the card
# recommends Q5_K_S/Q5_K_M and "Q6" — but DavidAU keeps the output tensor at
# fp16 and the MTP tensors at Q8_0, so his MTP-Q5_K_M is 19.73 GiB, +2.50 over
# this file. Headroom here is bounded between +2.00 GiB (q8_0 KV @196608 FIT
# on fable-fusion) and +4.00 GiB (q8_0 KV @262144 did NOT). MTP-Q5_K_S
# (19.21 GiB, +1.98) is the stretch pick IF this trial wins — expect the
# 262144 -> 196608 ladder. Q6_K (22.38) and Q8_0 (28.16) leave no context.
# IQ2_M is out: 2-bit destroyed API recall on tommy2 (office 0/9 unaided).
# The two "LOW-MTP" files reduce MTP precision = an untested third variable.
# MTP over "regular": this arch measures 78-80% draft acceptance on
# fable-fusion, far above the card's own 50% fall-back-to-regular threshold.
#
# TEMPLATE — THE STEP-0 GATE, AND IT FAILED. The embedded template carries
# raise_exception('System message must be at the beginning.')
# so Claude Code 500s on turn one. It is the QWEN3.8 template (reasoning_effort
# xhigh/medium/low, video tokens, image counting), NOT the Qwen3.6 one, so
# chat-templates/fable-711-256k.jinja is the WRONG patch to reuse. Extracted the
# embedded template and it diffs CLEAN against the proven
# chat-templates/qwen3.8-27b-nonfirst-system.jinja — that one guard line and a
# trailing newline, nothing else. Patched copy kept separately as
# chat-templates/turbo-fcf-735-882-256k.jinja so a re-pull of either repo cannot
# silently desync the other entry.
#
# TWO FLAGS DELIBERATELY OMITTED, both borrowed from the sibling `qwen38`
# entry (same Qwen3.8-27B base) and both to be tried ONLY after the A/B:
# --reasoning-format deepseek -- qwen38 sets it. FIRST thing to try if think
# tags leak into content; run bench/grill_textturn.py to detect that, the
# 125-turn grill scored aquila CLEAN while it leaked in real use
# (auto-memory/aquila-think-tag-leak-real-use.md).
# -b 2048 -ub 512 -- worth 2.7x PREFILL on laguna-s. Never tune a serving
# config on decode alone (auto-memory/laguna-s-2.1-cpu-spill.md).
#
# STORAGE: /mnt/models (HDD-backed SATA SSD, cold tier) because this is a
# TRIAL. NVMe is reserved for entries that swap in constantly; promote the
# weights there only if this displaces fable-fusion.
#
# WHAT IT MUST PROVE TO EARN A SLOT. We already serve Qwen3.8-27B three ways
# (`qwen38` llama.cpp Q5_K_M, `qwen38-awq`, `qwen38-mtp` vLLM), so "it is good"
# is not enough — it must either beat fable-fusion on KLAYOUT-UNAIDED +
# CLOSED-LOOP (the niche nothing else on this box fills) or beat qwen38 on
# coding. Grill those arms first; a full battery is hours.
cmd: |
/usr/bin/env
CUDA_DEVICE_ORDER=PCI_BUS_ID
${llama_bin}
-m /mnt/models/gguf/Qwen3.8-27B-TurboFCFusion-735-882-MTP-Q4_K_M.gguf
--mmproj /mnt/models/gguf/Qwen3.8-27B-TurboFCFusion-735-882-mmproj-BF16.gguf
--alias fable-turbo
--jinja --chat-template-file chat-templates/turbo-fcf-735-882-256k.jinja
-ngl 99 -c 262144 -fa on
-sm tensor
--spec-type draft-mtp --spec-draft-n-max 2
--temp 0.6 --top-p 0.95 --top-k 20 --min-p 0.0
--cache-type-k q4_0 --cache-type-v q4_0
--host 127.0.0.1 --port 9196 --parallel 1
proxy: http://127.0.0.1:9196