Kept / trial active in llama-swap.yaml
| Run | Score | tok/step | R5 | Log |
|---|---|---|---|---|
| No results-*.log found for this id/alias. | ||||
Trialled 2026-08-10, cyankiwi/ThinkingCap-Qwen3.6-27B-AWQ-INT4, 19.6 GiB,
Qwen3_5ForConditionalGeneration, 262144 native. Full 12-suite set x 2 repeats
in 111 min, 0 failed suites. Served as thinkingcap;
--tool-call-parser qwen3_coder --reasoning-parser qwen3, gate 4/4 including
the new leaked-control-token check.
| suite | combined | field position |
|---|---|---|
| coding R1-R4 | 45/46 | HIGHEST EVER HERE (next: 44/46 x3) |
| R5 agentic | 12/12 | tied best, 0 looped, 0 redundant calls |
| klayout from memory | 2/16 | mid (fable-fusion 5/16) |
| klayout + ref | 14/16 | joint 3rd — rep1 was 8/8, rep2 6/8, spread 2 |
| klvision | 10/12 | tied best, spread 0 |
| closed loop, NO ref | 8/8 | tied best — only fable-fusion also does this |
| closed loop + ref | 8/8 | tied best |
| office / office+ref | 16/18 / 18/18 | tied best assisted |
| cap-hits | 0 | tied best |
| throughput | 27 tok/s | SLOWEST on the box |
It closes the KLayout loop UNAIDED, 4/4 twice. Verified by re-executing the
artifact rather than trusting the grader: loop_array gives 12 polygons, area
48,000,000 DBU^2, bbox 17.0 x 12.0 um — identical to the hand-checked reference
in [[closed-loop-works-with-api-ref]]. It also reproduces fable-fusion's
signature shape — 8/8 on BOTH arms, i.e. **the API reference buys it nothing on
the loop because it already knows the API.** Two models on this box do that.
First model ever to pass expr_eval — but only once in two runs, at 9109
tokens and 324s. [[fable-fusion-regrilled]] records that task failing "for every
model on the box" and dismisses it as not a discriminator; that is no longer
strictly true. The honest statement is "the only model that can solve it, about
half the time, very expensively" — the difference between its two R1 repeats
(7/8 then 8/8, the only 8/8 in 24 recorded repeats) is entirely this task.
I ARGUED AGAINST RUNNING THIS TRIAL AND WAS WRONG. My reasoning: same
Qwen3.6-27B family as fable-711-gptq (44/46), so it would be "another ~44/46
dense 27B" winning no slot, and its token-efficient tag was unproven. Outcome:
it beats fable-711-gptq on coding (+1), the unaided loop (8/8 vs 2/8), office
(16/18 vs 6/18) and cap-hits (0 vs 6). The family-similarity argument predicted
the SPEED correctly and the CAPABILITY not at all.
The cost argument WAS right and still stands. The token-efficient /
efficient-thinking tags are not borne out:
thinkingcap 1722 median coding tok 27 tok/s
fable-711-gptq 787 28 tok/s <- same family, 2.2x leaner
gemma-awq 512 93 tok/s <- 3.4x leaner, 3.4x faster
R4: same 5/5 as gemma-awq for 1433 tok / 56.9s against 460 tok / 5.6s —
10x the wall clock for the same result. R5 median wall/task 24.2s.
Context: 262144 native, but only 131072 is servable here. KV costs 32
KiB/token — 16 of 64 layers are full_attention at head_dim 256 — so the full
window needs 4.00 GiB/GPU against the 2.49 GiB measured free; the pool tops out
near 155k tokens. Contrast [[gemma-4-26b-awq-vllm]], which serves its full
262144 because only 5 of its 30 layers are full-attention (20 KiB/token). Same
advertised window, 1.6x the real cost — see [[kv-sizing-full-attention-layers]].
Where it belongs. It is the strongest model here on capability and the
slowest on delivery. gemma-awq remains the driver ([[gemma-4-26b-awq-vllm]]);
this is the model to select by name when the answer matters more than the wait —
the role [[fable-fusion-regrilled]] holds for KLayout, which thinkingcap matches
on both loop arms while beating it on coding and office. **SERVED as
thinkingcap on port 9137 since 2026-08-10, but NOT promoted** — no client
defaults to it; select it by name.
"thinkingcap":
# STORAGE TIER: /mnt/ssk500 — DEMOTED from the NVMe 2026-08-17. The HF cache
# dir ~/.cache/huggingface/hub/models--cyankiwi--ThinkingCap-... is now a
# SYMLINK to /mnt/ssk500/models/, which is what makes vLLM's `serve <repo-id>`
# resolve to the slow disk. Move was FREE (checksum-identical copy already
# there, verified with rsync -aHc --dry-run before deleting the original).
# Cheap to demote precisely BECAUSE it is the slowest model here: disk is only
# ~46 s of its ~370 s vLLM TP=2 cold start (~12%) — flashinfer JIT, CUDA-graph
# capture and weight sharding dominate — and at 27 tok/s you always reach for
# it deliberately. To promote it back, replace the symlink with a real copy;
# do NOT rsync through the symlink or you overwrite the backup in place.
#
# THE HIGHEST-SCORING MODEL ON THE BOX, and the slowest. ThinkingCap-Qwen3.6-27B
# AWQ-INT4, 20 GiB. Coding 45/46 — one clear of the 44/46 held by three models —
# R5 12/12 with ZERO redundant calls, 8/8 on BOTH closed-loop arms, office+ref
# 18/18, and ZERO cap-hits in 24 suite runs. See auto-memory thinkingcap-awq-trial.
#
# PICK IT WHEN THE ANSWER MATTERS MORE THAN THE WAIT. 27 tok/s, the slowest
# here, and 1722 median coding tokens against gemma-awq's 512 at 93 tok/s —
# about 10x the wall clock for the same R4 result. Its `token-efficient` tag
# is NOT borne out: its own sibling fable-711-gptq is 2.2x leaner at the same
# speed.
#
# Like fable-fusion it closes the KLayout loop UNAIDED (4/4 twice, geometry
# verified by re-execution) and gains NOTHING from the API reference there —
# it already knows the API. Only these two models do that.
#
# --reasoning-parser qwen3 is required; the gate's leaked-control-token check
# confirms it (gemma-awq shipped a whole grill leaking <|channel> without one).
#
# 131072 is a HARDWARE limit, not a model one: native is 262144, but KV here
# costs 32 KiB/token (16 of 64 layers are full_attention at head_dim 256), so
# 262k would need 4.00 GiB/GPU against the 2.49 GiB measured free — the pool
# tops out near 155k tokens. Contrast gemma-awq, which serves its full 262144
# because only 5 of its 30 layers are full-attention (20 KiB/token). Same
# advertised window, 1.6x the real cost. See kv-sizing-full-attention-layers.
cmd: |
/usr/bin/env
HF_HOME=/home/giovanni/.cache/huggingface
CUDA_DEVICE_ORDER=PCI_BUS_ID
PATH=/home/giovanni/vllm-env/bin:/home/giovanni/.local/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
${vllm_bin} serve cyankiwi/ThinkingCap-Qwen3.6-27B-AWQ-INT4
--served-model-name thinkingcap
--tensor-parallel-size 2
--host 127.0.0.1 --port 9137
--max-model-len 131072
--max-num-seqs 8
--gpu-memory-utilization 0.90
--kv-cache-dtype fp8
--override-generation-config '{"temperature":1.0}'
--enable-auto-tool-choice
--tool-call-parser qwen3_coder
--reasoning-parser qwen3
proxy: http://127.0.0.1:9137