Models on server — ranked by capability

Updated 2026-11-11. Every tested model (webbench, grill, wall times) comes first; the llama-swap ranking follows, and the split entries (one model on both GPUs, two requests at once) are at the end.

Every tested model — webbench, grill and wall times

147 models with at least one measurement on this box: everything served by llama-swap.yaml or by a systemd unit, the cloud references, and trials whose weights are gone. 18 finished the full 134-task webbench (100 web/UI tasks + 14 React + 20 Next.js, strict pass, sample 1, model-default temperature); they are listed first, by score. A shorter run is shown as partial and is not comparable with a full score. Web wall is the summed generation time of the 134 tasks (grading excluded), so it shows what a full run costs on that model. Grill is the mean over every canonical 23-task run; Grill wall is lm_eval + rounds 1-5 per rep, from logs written since 2026-10-12 (older runs read "—"). Webbench scores come from the per-task score files, the same source as the compare pages. Results filed under the retired generic router names (agent, chat, coder …) are left out, because those names no longer identify one model. Hover a cell for its source.

# Model Where Engine Webbench (/134) Web wall Web tok/s Grill (/23) Grill wall R5 complete R5 looped Vision
1 claude-sonnet-5-5 cloud reference cloud API 129/134 0h43m 158 22.0/23 7m25s 6/6 0/6 44/52
2 strata-gyro-l systemd unit Strata 120/134 27h07m 32 22.0/23 71m20s 6/6 0/6 50/52
3 strata-gyro-m systemd unit Strata 111/134 13h08m 68 21.7/23 32m40s 6/6 0/6 52/52
4 strata-swift-iq3_xxs systemd unit Strata 107/134 8h58m 95 20.7/23 23m55s 5/6 0/6 50/52
5 strata-gyro-s systemd unit Strata 103/134 10h00m 79 22.0/23 28m42s 6/6 0/6 51/52
6 maya-glm-s24
maya		 systemd unit Maya 99/134 15h13m 11 20.0/23 80m48s 6/6 0/6 51/52
7 swift-sglang systemd unit SGLang 96/134 7h40m 74 21.3/23 22m05s 5/6 0/6 —
8 ariane
swift-fp4, qwen38-ariane, qwen38-webdev, swift15-nvfp4 …		 systemd unit vLLM 95/134 9h10m 66 22.0/23 — 5/6 0/6 49/52
9 esmeralda systemd unit vLLM 92/134 2h25m 93 22.0/23 17m33s 6/6 0/6 51/52
10 quasimodo systemd unit vLLM 91/134 2h38m 88 21.7/23 — 5/6 0/6 51/52
11 strata-iq2_xs systemd unit Strata 88/134 9h50m 91 22.0/23 27m02s 5/6 0/6 50/52
12 quasar not installed — 85/134 2h12m 89 22.0/23 16m49s 6/6 0/6 48/52
13 gemma-4-31b-qat systemd unit llama.cpp 75/134 7h19m 21 22.3/23 54m59s 6/6 0/6 49/52
14 gpt-oss-20b
gpt-oss-20b-mxfp4-4060ti-131k, gpt-oss-20b-128k		 llama-swap llama.cpp 66/134 2h55m 109 22.3/23 — — — —
15 nemotron-bolt
nemotron-3.5-bolt-74b-a9b-24gb-256k, bolt		 systemd unit llama.cpp 60/134 9h53m 57 22.2/23 23m29s 5/6 1/6 50/52
16 gemma-awq not installed — 44/134 0h56m 111 22.0/23 — 6/6 0/6 —
17 qwen38-coder-strata
strata-coder, coder-strata, strata-coder-iq1m		 llama-swap Strata 32/134 12h57m 71 21.0/23 — 5/6 0/6 49/52
18 darwin-split
darwin-35b-vision-x2, darwin-vision-x2		 systemd unit llama.cpp 26/134 1h42m 116 21.2/23 17m22s 6/6 0/6 52/52
19 maya-glm-xs
maya-xs		 systemd unit Maya 17/27 partial — 17 22.0/23 79m47s 5/6 0/6 49/52
20 qwen38-avalon-patched systemd unit llama.cpp 14/17 partial — 48 22.0/23 45m18s 6/6 0/6 —
21 bolt-q4kxl not installed — 21/34 partial — 27 21.7/23 61m10s 6/6 0/6 48/52
22 nemotron-nvfp4
nemotron-3.5-lightning-30b-a3b-nvfp4, nemotron		 systemd unit vLLM 4/22 partial — 180 21.3/23 — 4/6 1/6 —
23 kimi-distill
qwen3.6-35b-a3b-kimi-k2.6-distill-256k		 llama-swap llama.cpp 7/23 partial — 111 21.2/23 — 6/6 0/6 —
24 devstral-small-2 systemd unit llama.cpp 4/29 partial — 20 20.8/23 16m28s 5/6 0/6 45/52
25 qwen38-pi systemd unit llama.cpp 6/12 partial — 59 20.7/23 37m09s 6/6 0/6 50/52
26 clover-35b not installed — 15/25 partial — 63 20.0/23 41m21s 6/6 0/6 —
27 claude-haiku-5-5 cloud reference cloud API 90/100 partial — 250 — — — — —
28 gpt-oss-120b
gpt-oss-120b-mxfp4-cpumoe-131k		 llama-swap llama.cpp 13/26 partial — 19 — — — — —
29 ripple-65b-dflash2 not installed — 9/28 partial — 21 — — — — —
30 fable not installed — — — — 23.0/23 — — — —
31 gemma-4-31b-qat-vllm not installed — — — — 22.7/23 11m12s 4/6 0/6 47/52
32 muse
muse-glimmer-30b-nvfp4, muse-glimmer-nvfp4		 llama-swap vLLM — — — 22.7/23 — 6/6 0/6 51/52
33 swift-split
swift-1.5-qwen3.8-27b-256k-x2, swift15-x2		 llama-swap llama.cpp — — — 22.3/23 — 5/6 0/6 51/52
34 Kimi-K2.6-slow not installed — — — — 22.0/23 — 6/6 0/6 —
35 agentworld not installed — — — — 22.0/23 — — — —
36 coder-agentic
qwen3-coder-next-80b-256k		 llama-swap llama.cpp — — — 22.0/23 — 5/6 1/6 —
37 coder-prune-x2 not installed — — — — 22.0/23 — 5/6 1/6 —
38 fable-711-gptq systemd unit, not installed vLLM — — — 22.0/23 — 6/6 0/6 —
39 fable-fusion
qwen3.6-27b-fable-711-256k		 llama-swap llama.cpp — — — 22.0/23 — 6/6 0/6 —
40 fable-fusion-x2 not installed — — — — 22.0/23 — 6/6 0/6 —
41 fable-turbo
qwen3.8-27b-turbo-fcf-735-882-256k		 llama-swap llama.cpp — — — 22.0/23 — 5/6 0/6 —
42 gemma-4-31b-qat-nvfp4 not installed — — — — 22.0/23 11m31s 3/6 1/6 47/52
43 glm-flash-ck
glm-4.7-flash-awq-cyankiwi		 llama-swap vLLM — — — 22.0/23 — 6/6 0/6 —
44 laguna-xs not installed — — — — 22.0/23 — — — —
45 lupin systemd unit — — — — 22.0/23 41m26s 5/6 0/6 50/52
46 mellum systemd unit, not installed llama.cpp — — — 22.0/23 — 6/6 0/6 —
47 muse-glimmer
muse-glimmer-30b-131k		 llama-swap llama.cpp — — — 22.0/23 — 6/6 0/6 —
48 nemotron-lightning not installed — — — — 22.0/23 — 4/6 1/6 —
49 north-mini
north-mini-code-1.0-cohere2moe-488k		 llama-swap llama.cpp — — — 22.0/23 — 4/6 1/6 —
50 pitagora
thinkingcap-qwen38, thinkingcap-qwen3.8-27b-nvfp4a4		 llama-swap vLLM — — — 22.0/23 — 5/6 0/6 48/52
51 pocket-35b
pocket-35b-a3b-qwen3.5-256k		 llama-swap llama.cpp — — — 22.0/23 — 6/6 0/6 —
52 qnext
qwen3next-thinking-q3, qwen3-next-80b-thinking-q3		 llama-swap llama.cpp — — — 22.0/23 — 6/6 0/6 —
53 qwen3-coder-next-80b-128k not installed — — — — 22.0/23 — — — —
54 qwen3-coder-next-80b-160k not installed — — — — 22.0/23 — 6/6 0/6 —
55 qwen3-coder-next-80b-200k not installed — — — — 22.0/23 — 6/6 0/6 —
56 qwen3-vl-30b-256k not installed — — — — 22.0/23 — — — —
57 qwen3.6-35b-a3b-mtp-gguf-q4_k_m-256k not installed — — — — 22.0/23 — — — —
58 qwen38
qwen3.8-27b-256k		 llama-swap llama.cpp — — — 22.0/23 — 5/6 0/6 —
59 strata-gyro-m-cache not installed Strata — — — 22.0/23 32m20s 6/6 0/6 50/52
60 swift
swift-qwen3.8-27b-256k		 llama-swap llama.cpp — — — 22.0/23 — 5/6 0/6 —
61 tess-4-27b not installed — — — — 22.0/23 — — — —
62 thinkingcap
thinkingcap-qwen3.6-27b-256k		 llama-swap vLLM — — — 22.0/23 — — — 21/21
63 tommy
thomson-1.0-small-qwen35-256k		 llama-swap llama.cpp — — — 22.0/23 — 6/6 0/6 —
64 vision-coder not installed — — — — 22.0/23 — 6/6 0/6 —
65 darwin-35b
darwin-35b-a3b-opus-256k		 llama-swap llama.cpp — — — 21.7/23 — 6/6 0/6 —
66 muse-fast not installed — — — — 21.7/23 — 5/6 0/6 —
67 quasimodo-textonly not installed — — — — 21.7/23 — 6/6 0/6 —
68 swift15
swift-1.5-qwen3.8-27b-256k		 llama-swap llama.cpp — — — 21.7/23 — 6/6 0/6 —
69 tommy-split
thomson-1.0-small-qwen35-x2, tommy-x2		 llama-swap llama.cpp — — — 21.5/23 31m40s 6/6 0/6 50/52
70 darwin
darwin-35b-a3b-opus-vision-256k, darwin-35b-vision		 llama-swap llama.cpp — — — 21.3/23 — 6/6 0/6 52/52
71 euclide not installed — — — — 21.3/23 — 5/6 0/6 47/52
72 swift-temp1 not installed — — — — 21.3/23 — 5/6 0/6 —
73 bonsai-duo not installed — — — — 21.0/23 — 5/6 1/6 —
74 coder-agentic-q3
qwen3-coder-next-80b-q3		 llama-swap llama.cpp — — — 21.0/23 — 6/6 0/6 —
75 darwin-35b-notebook not installed — — — — 21.0/23 20m23s 6/6 0/6 —
76 euclide38
Euclide-Qwen38		 llama-swap SGLang — — — 21.0/23 19m05s 5/6 0/6 —
77 gemma-exl3 not installed — — — — 21.0/23 — 5/6 0/6 —
78 gemma12 not installed — — — — 21.0/23 — 5/6 0/6 —
79 gemma12-solo not installed — — — — 21.0/23 — 4/6 0/6 —
80 genesis-v5 not installed — — — — 21.0/23 — — — —
81 glm-reap-q4 not installed — — — — 21.0/23 — 6/6 0/6 —
82 kimi-distill-x2 not installed — — — — 21.0/23 — 6/6 0/6 —
83 kimi-k2.6-vision
qwopus-k2.6-ik, qwopus3.5-122b-a10b-kimi-k2.6-destill		 llama-swap llama.cpp — — — 21.0/23 — 6.0/6 0.0/6 —
84 laguna-s-iq2m not installed — — — — 21.0/23 — 6/6 0/6 —
85 north-mini-x2 not installed — — — — 21.0/23 — 6/6 0/6 —
86 ornith-15-35b
ornith-15-35b-a3b-256k		 llama-swap llama.cpp — — — 21.0/23 — 5/6 0/6 —
87 ornith-15-awq
ornith-1.5-35b-awq, ornith-35b-awq		 llama-swap vLLM — — — 21.0/23 — 6/6 0/6 48/52
88 pocket-35b-x2 not installed — — — — 21.0/23 — 6/6 0/6 —
89 pocket-vision
pocket-35b-vision		 llama-swap llama.cpp — — — 21.0/23 — 6/6 0/6 51/52
90 qwen3-coder systemd unit, not installed vLLM — — — 21.0/23 — 5/6 0/6 —
91 qwen3-instruct
qwen3-30b-a3b-instruct-2507-131k		 llama-swap vLLM — — — 21.0/23 — 4/6 0/6 —
92 qwen36-35b
qwen3.6-35b-128k		 llama-swap vLLM — — — 21.0/23 — 6/6 0/6 —
93 qwen38-awq
qwen3.8-27b-awq-196k		 llama-swap vLLM — — — 21.0/23 — 6/6 0/6 —
94 qwen38-flash-next not installed — — — — 21.0/23 — 5/6 0/6 49/52
95 qwen38-flash-text not installed — — — — 21.0/23 — 6/6 0/6 —
96 qwen38-mtp
qwen3.8-27b-awq-mtp		 llama-swap vLLM — — — 21.0/23 — 6/6 0/6 —
97 qwen38-strata-plus
strata-plus, qwen38-strata-iq3s, strata-iq3s		 llama-swap Strata — — — 21.0/23 — 5/6 0/6 49/52
98 qwen3vl-8b llama-swap llama.cpp — — — 21.0/23 — 4/6 0/6 —
99 tiel-coder
tiel-coder-35b-a3b-mtp-256k		 llama-swap llama.cpp — — — 21.0/23 — 6/6 0/6 50/52
100 tommy2
thomson-1.0-small-iq3xxs-4060ti-131k		 llama-swap llama.cpp — — — 21.0/23 — 6/6 0/6 —
101 bonsai2
ternary-bonsai-2-27b-256k		 llama-swap llama.cpp — — — 20.7/23 — 6/6 0/6 —
102 devstral not installed — — — — 20.7/23 — — — —
103 pocket-split
pocket-35b-vision-x2, pocket-vision-x2		 llama-swap llama.cpp — — — 20.7/23 — 6/6 0/6 51/52
104 qwen3vl
qwen3vl-30b-instruct		 llama-swap vLLM — — — 20.7/23 — 5/6 0/6 —
105 strata-coder-iq1m-tuned not installed Strata — — — 20.7/23 — 5/6 0/6 49/52
106 glm-flash-q5 not installed — — — — 20.5/23 — 6/6 0/6 —
107 darwin-35b-q5 not installed — — — — 20.3/23 — 5/6 0/6 —
108 qwen38-fast
qwen3.8-27b-fast-256k		 llama-swap llama.cpp — — — 20.3/23 — 6/6 0/6 —
109 qwen38-strata
strata, qwen38-flash-next-strata, qwen38-strata-iq4xs, strata-iq4xs …		 systemd unit Strata — — — 20.3/23 — 6/6 0/6 50/52
110 qwen38-strata-swift
strata-swift, swift-strata, strata-swift15-iq2xs		 llama-swap Strata — — — 20.3/23 — 5/6 0/6 49/52
111 kat-coder-fast
kat-coder-mtp-gguf, katmtp		 llama-swap llama.cpp — — — 20.2/23 — 6/6 0/6 —
112 qwen38-gsq
qwen3.8-27b-gsq-rco-iq3s-256k, gsq-rco		 llama-swap llama.cpp — — — 20.1/23 — 6/6 0/6 —
113 agents-a1-35b-256k not installed — — — — 20.0/23 — — — —
114 coder-prune not installed — — — — 20.0/23 — 5/6 1/6 —
115 glm-flash-awq
glm-flash		 systemd unit, not installed vLLM — — — 20.0/23 — 6/6 0/6 —
116 glm-reap not installed — — — — 20.0/23 — 6/6 0/6 —
117 laguna-s not installed — — — — 20.0/23 — 6/6 0/6 —
118 nemotron-nano not installed — — — — 20.0/23 — 5/6 0/6 —
119 qnext-fast
qwen3next-thinking, qwen3-thinking-80b		 llama-swap llama.cpp — — — 20.0/23 — 5/6 0/6 —
120 qwen3-vision systemd unit, not installed vLLM — — — 20.0/23 — 5/6 0/6 —
121 qwen3-vl-thinking llama-swap vLLM — — — 20.0/23 — 5/6 0/6 —
122 k2-horizon
k2-horizon-mova-36b-a4b, mova-36b		 llama-swap llama.cpp — — — 19.7/23 — 5/6 0/6 —
123 qwen38-strata-fast
strata-fast, qwen38-strata-iq2xs, strata-iq2xs		 llama-swap Strata — — — 19.7/23 — 5/6 0/6 49/52
124 bonsai
ternary-bonsai-27b-196k		 llama-swap llama.cpp — — — 19.5/23 — 6/6 0/6 —
125 tiel-coder-q5 not installed — — — — 19.5/23 — 6/6 0/6 —
126 flashtext not installed — — — — 19.3/23 — 6/6 0/6 —
127 qwen3.5-4b not installed — — — — 19.3/23 15m45s 5/6 1/6 —
128 bonsai-awq llama-swap vLLM — — — 19.0/23 — 6/6 0/6 —
129 gemma-4-12b-agentic-v2 not installed — — — — 19.0/23 — — — —
130 kat-coder
kat-coder-v2.5-dev-mtp-exl3-5bpw		 llama-swap vLLM — — — 19.0/23 — 6/6 0/6 —
131 qwen38-gittensor-sglang not installed SGLang — — — 19.0/23 32m26s 5/6 0/6 51/52
132 ministral-3-14b
ministral-3-14b-instruct-2512-128k		 llama-swap llama.cpp — — — 18.7/23 — 4/6 0/6 —
133 kimi-linear not installed — — — — 18.3/23 — 4/6 2/6 —
134 nex-n25-mini not installed — — — — 18.0/23 — 6/6 0/6 —
135 glm-grande not installed — — — — 17.0/23 — — — —
136 mistral-small-4-iq2xxs systemd unit, not installed — — — — 16.5/23 12m29s 5/6 0/6 33/52
137 xing4 not installed — — — — 16.5/23 — 2/6 3/6 —
138 llama33-70b not installed — — — — 15.0/23 — 1/6 5/6 —
139 qwen3-thinking
qwen3-30b-a3b-thinking-2507-196k		 llama-swap vLLM — — — 14.0/23 — 5/6 0/6 —
140 qwen38-exl3
qwen3.8-27b-exl3-6bpw		 llama-swap vLLM — — — 14.0/23 — 5/6 0/6 —
141 mimo-distill-9b not installed — — — — 13.5/23 — — — —
142 white-cat not installed — — — — 13.3/23 — 6/6 0/6 —
143 bonsai-fast not installed — — — — 11.0/23 — 5/6 0/6 —
144 kimi-vl not installed — — — — 10.0/23 — — — —
145 devstral2 not installed — — — — 2.0/23 — — — —
146 avalon not installed — — — — — — — — 50/52
147 strata-iq3_s systemd unit Strata — — — — — — — 49/52

2026-10-09: second RTX 5060 Ti installed — load test on the new pair

The RTX 4060 Ti in PCI slot 02:00 was replaced by a second RTX 5060 Ti 16 GB, so the box now runs two identical cards (16311 MiB each). Every entry was then loaded through llama-swap and sent one text request (plus one screenshot for vision entries), with per-card VRAM read straight after. 54 of 54 tested entries passed; no entry failed because of the new card itself. qwen38-coder-strata was benchmarked on its own: it loads with no parameter change (cold load 30 s) and decodes at 73.9-83.5 t/s across 5 prompts, 3-21% faster per prompt than on the old pair. Grill scores and tok/s elsewhere on this page were measured before the swap and are not re-measured by this test.

Removed from llama-swap.yaml the same day (7): aquila (owner request; already unservable since 2026-09-28, as no split fit 262k + vision; weights deleted); nex-n25-mini (owner request; died on its first request on the new cards; weights deleted); apodex (owner request; loaded, but rejected as a driver earlier; weights deleted); gpt-oss-120b (weights no longer on any disk); qwen38-flash-next (weights deleted 2026-10-05 to make room for the Strata Coder); qwen38-flash-text (same deleted weights as qwen38-flash-next); xing4 (weights no longer on any disk).

Not yet re-tested on the new pair (19): bonsai-awq, fable-711-gptq, glm-flash-awq, glm-flash-ck, glm-flash-sglang, glm53-coder, qwen3-coder, qwen3-instruct, qwen3-thinking, qwen3-vision, qwen3-vl-thinking, qwen36-35b, qwen38-awq, qwen38-mtp, qwen38-strata-coolbox, qwen3vl, strata-iq3, thinkingcap, vela. These are the vLLM, Strata and SGLang entries, which size their memory from what the card reports at start-up; their rows above still show figures measured on the old pair.

Per-entry results (54)
Entry Text Image 01:00 used (MiB) 02:00 used (MiB) First request (s)
qwen38 OK OK 15177 14979 52.0
qwen38-fast OK OK 13855 14761 135.4
coder-agentic OK — 14771 14937 56.6
coder-agentic-q3 OK — 15359 15359 86.8
fable-fusion OK OK 14757 13529 58.5
fable-turbo OK OK 14759 13531 75.5
fable-turbo-split OK OK 14727 13499 6.3
qwen38-gsq OK OK 14041 12813 46.7
qwen38-gsq1 OK — 15677 15 6.0
qwen38-gsq2 OK — 15 15677 6.1
pocket-35b OK — 12511 12511 208.5
pocket-vision OK OK 14641 12913 23.5
pocket-split OK OK 14161 12429 7.5
darwin-35b OK — 12637 12637 27.4
darwin-35b-solo OK — 14391 15 8.6
darwin OK OK 14749 13017 9.8
darwin-split OK OK 14997 14193 8.1
kimi-distill OK — 14637 15035 82.7
kimi-k2.6-vision OK OK 14809 14983 445.1
Kimi-K2.6-slow OK — 15407 15203 435.5
kolibri-1 OK — 14451 14911 376.7
ornith-15-35b OK OK 12931 14621 58.3
tiel-coder OK OK 15111 15459 58.9
tommy OK OK 13883 12679 65.5
tommy-split OK OK 15287 14083 17.4
tommy2 OK OK 15 13941 31.7
tommy1 OK OK 13941 15 6.6
gpt-oss-20b OK — 15 14971 57.1
north-mini OK — 14897 14837 55.3
nemotron-lightning OK — 13767 15225 64.0
muse-glimmer OK OK 12893 12707 57.4
qnext OK — 15729 15733 26.6
qnext-fast OK — 14387 14941 215.0
bonsai OK OK 15 15615 40.5
bonsai2 OK OK 11279 10211 24.9
swift OK OK 14423 13189 53.4
swift15 OK OK 14291 13055 49.6
swift-split OK OK 15283 14047 6.5
agnes OK — 12883 13267 178.2
glm-ocr OK OK 15 4007 5.3
mellum OK — 15 14387 71.6
qwen3vl-8b OK OK 15 12109 34.4
kat-coder-fast OK — 13927 13927 76.9
k2-horizon OK — 15511 15007 79.8
ministral-3-14b OK OK 12547 11377 99.9
gemma12 OK — 15 14767 297.9
kat-coder OK — 14186 13429 95.5
gemma-exl3 OK — 14228 13457 113.8
qwen38-exl3 OK — 14114 13547 77.1
qwen38-strata OK OK 15602 15724 65.2
qwen38-strata-plus OK OK 15572 15694 175.6
qwen38-strata-fast OK OK 15504 15546 106.4
qwen38-strata-swift OK OK 15504 15546 121.9
ornith-15-awq OK OK 14813 14815 330.5

VRAM is read after the last request, so it includes the lazy cuBLAS workspace and, for vision entries, the image encoder's buffer. First request time includes the cold load. Raw logs: bench/gpu-swap-load-sweep-2026*.jsonl.

Strata-engine entries — Qwen3.8-Flash-Next on both cards, KV from RAM

These five run on a different engine from everything else on this page: github.com/Niko1221/Strata (engine 0.1.40), not llama.cpp or vLLM. Strata serves Qwen3.8-Flash-Next across both cards with a layer split and the KV cache streamed from host RAM (--kv-resident), so a 262k-context window fits in far less VRAM than a native server could hold. The five entries are the same engine at five weights: Unsloth's ~4-bit UD-IQ4_XS (qwen38-strata; added 2026-10-07, both cards plus a RAM budget, ~35-40 t/s, grill not yet complete), ISTA-DASLab's 3.5-bit IQ3_S (qwen38-strata-plus; added 2026-10-07, both cards plus a RAM budget, ~41-58 t/s, 21/23 in all three grill reps -- the best Strata coding score -- R5 16/18 with zero loops, vision 49/52), the unpruned IQ2_XS (qwen38-strata-fast, the fastest at ~75-84 t/s), the Swift 1.5 fine-tune (qwen38-strata-swift), and the Coder (qwen38-coder-strata) -- ISTA-DASLab's Qwen3.8-Flash-Next-GSQ-RCO-Coder-GGUF, a pruned IQ1_M that keeps 256 of each layer's 512 routed experts (chosen on code, tool use and vision), so it is smaller and faster but weaker outside code and in CJK.

Grilled 2026-10-05, three serial reps each at xhigh reasoning effort. The pruned Coder beats the unpruned model on the coding grill: 21.0/23 against Swift's 20.3 and IQ2_XS's 19.7, with zero loops in all three reps of all three entries. Vision is 49/52 across the board. Strata serves one request at a time behind a FIFO (no concurrency), so these are serial grills and the scores are uncontended. Decode is ~68-78 t/s with ~2300 t/s prefill. All three are trials, not promoted drivers; they are mutually exclusive (each holds both cards) and strata.service is deliberately not installed so it cannot race the llama-swap entries. Each entry's full caveats are in its llama-swap.yaml comment block and in auto-memory/.

Model Params Context Vision tok/s Grill (/23) Grill wall R5 complete R5 looped Verdict
qwen38-coder-strata
strata-coder, coder-strata		 — 256k 49/52 69 21.0/23 — 5.3/6 0.0/6 TUNED CONFIG since 2026-10-06 (strata-coder-iq1_m-tuned.json; the stock strata-coder-iq1_m.json is kept for A/B). Two changes, A/B'd with bench/strata-coder-ab.py (one sample per arm, same 5-prompt greedy workload): 1. t
qwen38-strata-plus
strata-plus, qwen38-strata-iq3s, strata-iq3s		 — 256k 49/52 32 21.0/23 — 5.3/6 0.0/6 GRILLED 2026-10-07 (canonical 3x + vision52 3x, xhigh, serial, :9215 direct, label strata-iq3s, bench/grill-strata-iq3s-3x.sh): Grill /23 21, 21, 21 = 21.0 -- the BEST Strata build (IQ2_XS 19.7, Coder 20.67, Swift 20.3)
qwen38-strata-swift
strata-swift, swift-strata		 — 256k 49/52 68 20.3/23 — 5.0/6 0.0/6 TRIAL 2026-10-05. Sibling of `qwen38-strata-iq2xs` / `qwen38-strata-swift` above: SAME Strata engine (v0.1.39 at install, 0.1.40 since 2026-10-06 -- see qwen38-strata-iq2xs; 0.1.39 was the upgrade from v0.1.31 for this e
qwen38-strata
strata, qwen38-flash-next-strata, qwen38-strata-iq4xs, strata-iq4xs, unsloth-iq4xs-strata		 — 256k 50/52 34 20.3/23 — 6.0/6 0.0/6 GRILLED 2026-10-07/08 (canonical 3x + vision52 3x, xhigh, serial, :9214 direct, label strata-iq4xs-unsloth, bench/grill-strata-iq4xs-3x.sh; rep 1 2026-10-07 morning, reps 2-3 re-run overnight after the first attempt was
qwen38-strata-fast
strata-fast, qwen38-strata-iq2xs, strata-iq2xs		 — 256k 49/52 53 19.7/23 — 5.7/6 0.0/6 TUNED CONFIG since 2026-10-06 (strata-iq2_xs-tuned.json; stock kept for A/B): explicit layer split 27 + --trim-stage-weights (+ --expert-profile-save, fit_max_tokens; profile UNCHANGED -- the shipped 48x512 ranking is ge

Members here: qwen38-coder-strata, qwen38-strata-plus, qwen38-strata-swift, qwen38-strata, qwen38-strata-fast. Selected by id (the three qwen38-*-strata entries), ordered by grill score. These rows also appear in the ranked table below -- this section is a view, not a second measurement. Logs: bench/grill-strata-*-3x.sh wrote bench/results-strata-*-<stamp>.log per grill rep and bench/results-vision-strata-*-rep<N>-<stamp>.log per vision rep.

llama-swap entries — ranked

79 active model entries from llama-swap.yaml (rejected, retired, and non-chat models excluded). Ranked from best to last: vision-capable models first, then by fastest decode tok/s, then by grill score (/23). Display names are the llama-swap.yaml entry names, with their aliases underneath. Parameter count (total weights, parsed from the GGUF header — not "active" params for a MoE), context window served (read off the entry's own command, so it is what a request actually gets, not the model's ceiling), median decode tok/s (from grill logs, representative not precise), and the verdict behind each one. A row with no measured speed — a usage-less backend like TabbyAPI/EXL3 reports none at all — shows "—" and is placed at the speed of the models it scored like, rather than sorted as if it ran at 0 tok/s. The Grill and both R5 figures are means across every run of that model, to one decimal: a single run is a sample, so it reads as one, and hovering any of them lists the runs behind it. A model whose runs disagree is shown as its average, never as its best. The FreeToken models are served by systemd units rather than by yaml entries, so they are not ranked in this table — they have their own section at the bottom, followed by two card-pool sections that split these entries by the GPU each one pins: the ones that stay usable on the other card while a FreeToken model is running, and the ones that take the FreeToken driver's own card. Both are views of this table, not second opinions on it — as are the Strata-engine section at the top and the *-split sections at the end, whose entries are counted above and appear in this table as well.

# Model Backend Status Params Context tok/s Grill (/23) Grill wall Webbench (/134) Web wall R5 complete R5 looped Vision Verdict
1 qwen3vl
qwen3vl-30b-instruct		 vLLM Promoted — 175k 138 20.7/23 — — — 5.0/6 0.0/6 yes remodlai build of the SAME base as qwen3-vision above (Qwen3-VL-30B-A3B-Instruct), but a DIFFERENT quantizer — and it grills BETTER than the QuantTrio build. The QuantTrio `qwen3-vision` comment above calls that build "t
2 tommy-split
thomson-1.0-small-qwen35-x2, tommy-x2		 llama.cpp Active 34.7B 224k 137 21.5/23 31m40s — — 5.8/6 0.0/6 50/52 The `darwin-split` shape applied to tommy: the SAME dual-card Q4_K_M entry with `--parallel 2` and a doubled `-c`, so one request gets both cards AND a spare slot. Added 2026-09-30, once darwin-split showed the shape was
3 qwen3-vl-thinking vLLM Active — 128k 137 20.0/23 — — — 5.0/6 0.0/6 yes THE VISION / KLAYOUT PICK. The ONLY model on this box that closes the KLayout loop: 4/4 code + 4/4 vision over 2 repeats with zero runaways, vs qwen3-vision 2/4 and fable-711-gptq 1/4 (auto-memory closed-loop-works-with-
4 qwen3-vision vLLM Active — 172k 130 20.0/23 — — — 5.0/6 0.0/6 yes WINDOW 176128 restored 2026-08-18 when vLLM was ROLLED BACK to 0.26.0. Under 0.27.1 this entry missed by ~10 MB ("176128 needs 4.03 GiB KV, available 4.02") and had to run at 172032. Raising utilization was NOT an option
5 darwin-split
darwin-35b-vision-x2, darwin-vision-x2		 llama.cpp Driver 34.7B 192k 110 21.2/23 17m22s 26/134 1h42m 6.0/6 0.0/6 52/52 TWO SLOTS ON BOTH CARDS — the `darwin` entry above with `--parallel 2`. THE DAILY DRIVER since 2026-10-01 — re-instated as the `claude-local` default, and the reason `model-router` and the darwin-solo1/solo2 twins were
6 tommy
thomson-1.0-small-qwen35-256k		 llama.cpp Kept / trial 34.7B 256k 108 22.0/23 — — — 6.0/6 0.0/6 yes thomsonreuters/Thomson-1.0-Small — TRIAL 2026-08-26. Thomson Reuters' "SovereignAI" value-aligned model. SAME arch as qwen36-35b: Qwen3.6-35B-A3B (qwen3_5_moe, Qwen3_5MoeForConditionalGeneration), 35B total / 3B active,
7 pocket-split
pocket-35b-vision-x2, pocket-vision-x2		 llama.cpp Active 34.7B 128k 95 20.7/23 — — — 5.7/6 0.0/6 51/52 The `darwin-split`/`tommy-split` shape applied to pocket-vision, built 2026-09-30 on owner request. Same weights, same mmproj, same patched template: the ONLY change from pocket-vision is `--parallel 2`.
8 nemotron-bolt
nemotron-3.5-bolt-74b-a9b-24gb-256k, bolt		 llama.cpp Active 74.0B 256k 94 22.2/23 23m29s 60/134 9h53m 5.2/6 0.8/6 50/52 ADDED 2026-10-17. NVIDIA Nemotron 3.5 Bolt 74B-A9B 24GB — NVIDIA's OWN llama.cpp GGUF (nvidia/NVIDIA-Nemotron-3.5-Bolt-74B-A9B-24GB-GGUF, 20.5 GiB): Super-3.5-VL-120B compressed (Star Elastic) to 74B/9B-active, experts Q
9 quasimodo vLLM Kept / trial — 128k 92 21.7/23 — 91/134 2h38m 5.2/6 0.3/6 51/52 TRIAL 2026-10-11, GRILLED 3x WITH VISION LOADED + parser v2: 21/22/22 = 21.67/23, R5 6/5/6 = 5.67/6 (1 loop: search_navigate rep 2), vision 51/52 x3 — `muse` + Meta's DFlash drafter (fp8, 7 draft tokens) + the muse_glimm
10 pocket-vision
pocket-35b-vision		 llama.cpp Kept / trial 34.7B 256k 92 21.0/23 — — — 6.0/6 0.0/6 51/52 VISION TWIN of pocket-35b (owner request 2026-09-26), built exactly the way `darwin` twins `darwin-35b`: same POCKET-35B Q4_K_M weights + the mmproj of the Qwen3.5-35B-A3B vision tower that `darwin` already serves (pocke
11 darwin
darwin-35b-a3b-opus-vision-256k, darwin-35b-vision		 llama.cpp Promoted 34.7B 256k 89 21.3/23 — — — 6.0/6 0.0/6 52/52 PROMOTED to driver 2026-09-25 — owner's DAILY DRIVER. Grilled on every axis: coding/agentic mean 21.3/23 + R5 6/6 all 3 reps (matches darwin-35b's own 21.7/23 — the --tensor-split split mode costs speed, NOT coding quali
12 qwen36-35b vLLM Kept / trial — 176k 88 21.0/23 — — — 6.0/6 0.0/6 yes RECONNECTED 2026-08-11. Trialled 2026-08-09 and left unserved pending a promotion call; this entry makes it reachable again WITHOUT retiring anything. QuantTrio/Qwen3.6-35B-A3B-AWQ, 24 GB, arch Qwen3_5MoeForConditionalGe
13 tommy2
thomson-1.0-small-iq3xxs-4060ti-131k		 llama.cpp Active 34.7B 128k 87 21.0/23 — — — 6.0/6 0.0/6 yes SINGLE-CARD CLONE of tommy on the 4060 Ti — runs IN PARALLEL with the FreeToken driver on the 5060 Ti (gemma4-ft / qwen36-ft). Same model (thomsonreuters/Thomson-1.0-Small, Qwen3.6-35B-A3B) + same patched family template
14 ornith-15-35b
ornith-15-35b-a3b-256k		 llama.cpp Kept / trial 35.5B 256k 85 21.0/23 — — — 5.0/6 0.0/6 yes ornith-ai/Ornith-1.5-35B-A3B-GGUF:Q4_K_M (20.22 GiB) + mmproj-BF16 (0.84 GiB) — TRIAL 2026-08-21. Ornith-1.5-35B-A3B, arch qwen3_5_moe (Qwen3_5MoeForConditionalGeneration) — SAME family as pocket-35b / qwen36-35b. Real s
15 ornith-15-awq
ornith-1.5-35b-awq, ornith-35b-awq		 vLLM Promoted — 256k 83 21.0/23 — — — 6.0/6 0.0/6 48/52 PROMOTED to concurrent driver 2026-09-25. Grill: 21/23 coding, R5 0-loop x3 (a 1/6 loop in the n=1 battery did not reproduce), realcase 7/7 clean @32k, office 9/9, vision52 48/52, klvision 5/6, closedloop-ref 4/4, 0 leak
16 tiel-coder
tiel-coder-35b-a3b-mtp-256k		 llama.cpp Promoted 35.5B 256k 78 21.0/23 — — — 5.8/6 0.0/6 50/52 PROMOTED to driver 2026-09-23. n=1 caveat RESOLVED (5 canonical runs): Grill 21.0/23, R5 5.8/6 with ZERO loops in ALL FIVE / 0% redundant, realcase 7/7, klvision 5/6, ~97.6 t/s. See auto-memory/tiel-coder-35b-a3b-mtp-tri
17 ariane
qwen38-ariane, qwen38-webdev, swift-fp4, swift15-nvfp4, swift-1.5-qwen3.8-27b-nvfp4, gpt-ariane		 vLLM Kept / trial — 160k 73 22.0/23 — 95/134 9h10m 5.0/6 0.0/6 49/52 RENAMED 2026-10-19 swift-fp4 -> ariane (owner, after the ESA Ariane rocket; archimede was used for a few hours, then dropped). swift-fp4 kept as an alias. qwen38-webdev = descriptive alias (owner 2026-10-19): best local
18 qwen38-coder-strata
strata-coder, coder-strata		 llama.cpp Kept / trial — 256k 69 21.0/23 — 32/134 12h57m 5.3/6 0.0/6 49/52 TUNED CONFIG since 2026-10-06 (strata-coder-iq1_m-tuned.json; the stock strata-coder-iq1_m.json is kept for A/B). Two changes, A/B'd with bench/strata-coder-ab.py (one sample per arm, same 5-prompt greedy workload): 1. t
19 qwen38-strata-swift
strata-swift, swift-strata		 llama.cpp Kept / trial — 256k 68 20.3/23 — — — 5.0/6 0.0/6 49/52 TRIAL 2026-10-05. Sibling of `qwen38-strata-iq2xs` / `qwen38-strata-swift` above: SAME Strata engine (v0.1.39 at install, 0.1.40 since 2026-10-06 -- see qwen38-strata-iq2xs; 0.1.39 was the upgrade from v0.1.31 for this e
20 pitagora
thinkingcap-qwen38, thinkingcap-qwen3.8-27b-nvfp4a4		 vLLM Kept / trial — 125k 64 22.0/23 — — — 5.7/6 0.0/6 48/52 TRIAL 2026-10-11, GRILLED 3x 2026-10-11: 21/22/23 = 22.0/23, R5 6/6/5 = 5.67/6 with ZERO loops, vision52 47.67/52 — bottlecapai/ThinkingCap-Qwen3.8-27B-NVFP4A4-AWQ on vLLM 0.31.0 TP=2, decode 50-56 t/s prose (code up to
21 strata-swift-iq3_xxs llama.cpp Kept / trial — 196k 56 20.7/23 23m55s 107/134 8h58m 5.7/6 0.0/6 50/52 TRIAL 2026-10-02, NOT RELOADED INTO llama-swap YET (no SIGHUP sent). Sibling of `qwen38-strata-iq2xs` above: SAME engine, SAME quant (IQ2_XS), SAME 262144 ctx / vision / KV-from-RAM config shape -- only the WEIGHTS diffe
22 muse-glimmer
muse-glimmer-30b-131k		 llama.cpp Kept / trial 27.9B 128k 55 22.0/23 — — — 6.0/6 0.0/6 yes meta-models/Muse-Glimmer-30B-GGUF — TRIAL 2026-08-11. Meta Superintelligence Labs' first open-weights model, Apache 2.0, distilled from Muse Spark 1.2. Dense causal transformer: ~28B decoder + ~1.8B ViT-G/14 perception e
23 qwen38-strata-fast
strata-fast, qwen38-strata-iq2xs, strata-iq2xs		 llama.cpp Kept / trial — 256k 53 19.7/23 — — — 5.7/6 0.0/6 49/52 TUNED CONFIG since 2026-10-06 (strata-iq2_xs-tuned.json; stock kept for A/B): explicit layer split 27 + --trim-stage-weights (+ --expert-profile-save, fit_max_tokens; profile UNCHANGED -- the shipped 48x512 ranking is ge
24 qwen38-gsq
qwen3.8-27b-gsq-rco-iq3s-256k, gsq-rco		 llama.cpp Kept / trial 27.3B 256k 49 20.1/23 — — — 5.8/6 0.0/6 yes TRIAL 2026-09-09 — ISTA-DASLab/Qwen3.8-27B-GSQ-RCO-GGUF, file Qwen3.8-27B-GSQ-RCO-IQ3_S-mtp.gguf (3.50 bpw, 11.29 GiB) + the repo's own mmproj-Qwen3.8-27B-BF16 (0.87 GiB). Weights on /mnt/models (cold tier, trial).
25 qwen3vl-8b llama.cpp Active 8.2B 128k 49 21.0/23 — — — 4.0/6 0.0/6 yes Qwen3-VL-8B-Instruct (Q4_K_M, 4.7 GB + 1.1 GB mmproj). Added 2026-08-23 as the smallest capable VISION model for co-residency — the 4060 Ti swap pool (mellum, gemma12, bonsai, glm-ocr) has no vision model of its own; thi
26 swift15
swift-1.5-qwen3.8-27b-256k		 llama.cpp Active 27.3B 256k 46 21.7/23 — — — 6.0/6 0.0/6 yes ukisai/Swift-1.5-Qwen3.8-27B-GGUF — wired 2026-09-28 on owner instruction, the same day the repo appeared. GRILLED 2026-09-29, 3 reps: mean 21.7/23, and R5 6/6 COMPLETE WITH ZERO LOOPS IN ALL THREE REPS.
27 swift-split
swift-1.5-qwen3.8-27b-256k-x2, swift15-x2		 llama.cpp Active 27.3B 160k 45 22.3/23 — — — 5.0/6 0.0/6 51/52 TWO SLOTS ON BOTH CARDS — `swift15` (Swift-1.5-Qwen3.8-27B) with `--parallel 2`. Wired 2026-10-01, modelled on `tommy-split`, which is the proof that the exact shape works on this box: `-sm tensor` + `--mmproj` + `--para
28 swift
swift-qwen3.8-27b-256k		 llama.cpp Kept / trial 27.3B 256k 44 22.0/23 — — — 5.0/6 0.0/6 yes THIS ENTRY IS SWIFT 1.0 — the 2026-09-18 build, 18,024,380,576 B / ad5811e2…. The 2026-09-28 re-merge is the SEPARATE `swift15` entry below; the two are kept side by side for an A/B, so check which name you are calling
29 qwen38-mtp
qwen3.8-27b-awq-mtp		 vLLM Kept / trial — 192k 39 21.0/23 — — — 6.0/6 0.0/6 yes TRIAL 2026-08-24 — shawnw3i/Qwen3.8-27B-AWQ-MTP: the SAME Qwen3.8-27B AWQ INT4 as `qwen38-awq`, plus a REAL multi-token-prediction draft head. Probed before downloading: 15 `mtp.*` tensors (mtp.fc + a full single-layer b
30 fable-turbo-split
qwen3.8-27b-turbo-fcf-735-882-x2, fable-turbo-x2		 llama.cpp Kept / trial 27.3B 128k 37 — — — — — — yes THE FIT IS TIGHTER THAN swift-split's: this file is 18,498,573,856 B vs Swift 1.5's 17,442,399,936 B (+0.98 GiB, ~+0.5 GiB per card under tensor split). MEASURED 2026-10-07, cold loads against verified-empty cards: -c 32
31 muse
muse-glimmer-30b-nvfp4, muse-glimmer-nvfp4		 vLLM Kept / trial — 128k 36 22.7/23 — — — 6.0/6 0.0/6 51/52 TRIAL 2026-10-09 — write_file double-escaping FIXED 2026-10-11 by the muse_glimmer_fixed parser plugin: the model still writes its FIRST file JSON-escaped without quotes (30 of 40 seeds at the probed step), the plugin de
32 bonsai
ternary-bonsai-27b-196k		 llama.cpp Active 26.9B 192k 35 19.5/23 — — — 6.0/6 0.0/6 yes THINKING MODEL: it spent 1,807 tokens answering "Blue" to a one-word vision question and ~1,300 tokens on a simple coding task. Raw decode is ~37 tok/s, close to gemma12's 42, but time-to-answer is ~4x longer. Budget acc
33 ministral-3-14b
ministral-3-14b-instruct-2512-128k		 llama.cpp Kept / trial 13.5B 128k 35 18.7/23 — — — 4.0/6 0.0/6 yes TRIAL, added 2026-09-23. mistralai/Ministral-3-14B-Instruct-2512 (unsloth Q6_K GGUF, 11.1 GiB + 839 MB mmproj-BF16) -- the largest Ministral that fits this box. Ministral-3 (Dec 2025): ~14B dense, Pixtral VISION tower (0
34 bonsai2
ternary-bonsai-2-27b-256k		 llama.cpp Active 26.9B 256k 34 20.7/23 — — — 6.0/6 0.0/6 yes prism-ml/Ternary-Bonsai-2-27B-gguf, PQ2_0 band (7.21 GB, 2.13 bpw). Added 2026-09-18. NOT a requant of the `bonsai` entry above — Bonsai 2 is a NEW model on a NEW base (Qwen/Qwen3.8-27B, 64 blocks, 27.36B params) whose w
35 qwen38-strata
strata, qwen38-flash-next-strata, qwen38-strata-iq4xs, strata-iq4xs, unsloth-iq4xs-strata		 llama.cpp Kept / trial — 256k 34 20.3/23 — — — 6.0/6 0.0/6 50/52 GRILLED 2026-10-07/08 (canonical 3x + vision52 3x, xhigh, serial, :9214 direct, label strata-iq4xs-unsloth, bench/grill-strata-iq4xs-3x.sh; rep 1 2026-10-07 morning, reps 2-3 re-run overnight after the first attempt was
36 fable-turbo
qwen3.8-27b-turbo-fcf-735-882-256k		 llama.cpp Kept / trial 27.3B 256k 33 22.0/23 — — — 5.0/6 0.0/6 yes Q6_K TRIAL CLOSED 2026-09-17 — REVERTED to MTP-Q4_K_M at -c 262144. The bigger quant was wired in on instruction and then grilled against this entry's own Q4_K_M. It bought nothing and cost plenty: 196608 Q6_K R1 7
37 fable-fusion
qwen3.6-27b-fable-711-256k		 llama.cpp Active 27.3B 256k 32 22.0/23 — — — 6.0/6 0.0/6 yes DavidAU Qwen3.6-27B-Fable-Fusion-711-Uncensored-Heretic-NM-DAU-NEO-MAX-MTP (GGUF 17.2GiB) — a multi-stage fine tune + merge over 7 base models on Qwen/Qwen3.6-27B, Heretic-decensored (ARA method: refusals 4/100 vs the ba
38 qwen38-fast
qwen3.8-27b-fast-256k		 llama.cpp Active 27.3B 256k 32 20.3/23 — — — 6.0/6 0.0/6 yes THE QUANT TWIN OF `qwen38`. Every flag is byte-identical -- same template, same 262144, same q4_0 KV, same MTP head, same mmproj, same 52,48 split. THE ONLY DIFFERENCE IS THE WEIGHTS QUANT, deliberately, so that anything
39 qwen38-strata-plus
strata-plus, qwen38-strata-iq3s, strata-iq3s		 llama.cpp Kept / trial — 256k 32 21.0/23 — — — 5.3/6 0.0/6 49/52 GRILLED 2026-10-07 (canonical 3x + vision52 3x, xhigh, serial, :9215 direct, label strata-iq3s, bench/grill-strata-iq3s-3x.sh): Grill /23 21, 21, 21 = 21.0 -- the BEST Strata build (IQ2_XS 19.7, Coder 20.67, Swift 20.3)
40 qwen38
qwen3.8-27b-256k		 llama.cpp Active 27.3B 256k 31 22.0/23 — — — 5.0/6 0.0/6 yes Qwen3.8-27B. REPLACED 2026-08-17 (same day it was installed): was unsloth's UD-Q5_K_XL at 196608 / q8_0 KV / no MTP, now JonathanColetti's Heretic-abliterated Q5_K_M with the MTP head retained. Same base model (Qwen/Qwen
41 bonsai-awq vLLM Active — 240k 30 19.0/23 — — — 6.0/6 0.0/6 yes RECONNECTED 2026-08-11, trialled 2026-08-10. prism-ml/Ternary-Bonsai-27B-AWQ-4bit, 18 GB — the smallest competitive footprint on the box.
42 fable-711-gptq vLLM Active — 152k 28 22.0/23 — — — 6.0/6 0.0/6 yes The most REPEATABLE model ever measured on this box — spread 0 across all eleven suites, 28/29 coding twice — but DENSE, so ~28 tok/s, a 3x penalty against glm-flash for a 2-task edge. Its closed-loop 6/8 was refuted (0/
43 qwen38-awq
qwen3.8-27b-awq-196k		 vLLM Kept / trial — 192k 27 21.0/23 — — — 6.0/6 0.0/6 yes vLLM COUNTERPART TO `qwen38`. NOT the same weights: cyankiwi's quant is of the BASE Qwen/Qwen3.8-27B (base_model_name in config.json), whereas the llama.cpp `qwen38` serves the UNCENSORED fine-tune of the same base. So t
44 thinkingcap vLLM Kept / trial — 128k 27 22.0/23 — — — — — 21/21 THE HIGHEST-SCORING MODEL ON THE BOX, and the slowest. ThinkingCap-Qwen3.6-27B AWQ-INT4, 20 GiB. Coding 45/46 — one clear of the 44/46 held by three models — R5 12/12 with ZERO redundant calls, 8/8 on BOTH closed-loop ar
45 kimi-k2.6-vision
qwopus-k2.6-ik, qwopus3.5-122b-a10b-kimi-k2.6-destill		 llama.cpp Kept / trial 124.6B 256k 15 21.0/23 — — — 6.0/6 0.0/6 yes TRIAL 2026-09-15 — KeinNiemand/Qwopus3.5-122B-A10B-Kimi-K2.6-destill-healed- abliterated-IK_GGUF:IQ2_KL. A community Kimi-K2.6 reasoning distill over Qwen3.5-122B-A10B (~10B active MoE), IQ2_KL ~2.69 bpw, 44.2 GiB weight
46 qwen38-exl3
qwen3.8-27b-exl3-6bpw		 TabbyAPI Active — 192k — 14.0/23 — — — 5.0/6 0.0/6 yes FIRST EXL3 ENTRY ON THE BOX — TabbyAPI + ExLlamaV3 backend, neither llama.cpp nor vLLM. Wired exactly like the vLLM entries: llama-swap owns the lifecycle, so selecting this model EVICTS the resident one instead of fight
47 tommy1
thomson-1.0-small-iq2m-5060ti-131k		 llama.cpp Active 34.7B 128k — — — — — — — yes SINGLE-CARD CLONE of tommy2 on the 5060 Ti, added 2026-09-30 — the exact mirror of the entry above, so the tommy pair can occupy BOTH cards the same way qwen38-gsq1/gsq2 do. Byte-identical cmd except CUDA_VISIBLE_DEVICES
48 vela
qwen36-vela, qwen36-a3b		 vLLM Active — 128k — — — — — — — yes NOT LOADED / UNTESTED (2026-10-20). Added as an install only; no standalone load test yet, no grill. nvidia/Qwen3.6-35B-A3B-NVFP4 (modelopt NVFP4, 22 GB on /mnt/ssk500/models/Qwen3.6-35B-A3B-NVFP4). Arch qwen3_5_moe: 40
49 nemotron-nvfp4
nemotron-3.5-lightning-30b-a3b-nvfp4, nemotron		 vLLM Kept / trial — 256k 171 21.3/23 — 4/22 partial — 4.0/6 1.0/6 0 TRIAL 2026-10-09 — nvidia's own modelopt checkpoint of the SAME model as the Q5_K_M GGUF nemotron-lightning (entry + weights removed 2026-10-10: same quality at half the speed), served by vLLM TP=2 on the 2x RTX 5060 Ti
50 qwen3-thinking
qwen3-30b-a3b-thinking-2507-196k		 vLLM Kept / trial — 128k 141 14.0/23 — — — 5.0/6 0.0/6 0 Qwen3-30B-A3B-Thinking-2507 (Alibaba, the July-2025 "2507" thinking refresh of the Qwen3-30B-A3B MoE), served as a vLLM AWQ via the QuantTrio build. TRIAL 2026-08-20: the box's one open gap is a FAST TEXT-ONLY THINKING M
51 qwen3-coder vLLM Kept / trial — 200k 133 21.0/23 — — — 5.0/6 0.0/6 0 The original vLLM office driver (vllm-tp2.service). Text-only, 200k ctx, ~141 tok/s. Kept because kimi-code/pi/omp reference it by this exact id — auto-memory vllm-office-server-client-wiring.
52 darwin-35b
darwin-35b-a3b-opus-256k		 llama.cpp Promoted 34.7B 256k 103 21.7/23 — — — 6.0/6 0.0/6 0 PROMOTED to driver 2026-09-24. FINAL-Bench/Darwin-35B-A3B-Opus (bartowski GGUF, Q4_K_M 21.4GiB) — the PARENT model POCKET-35B was quantized from. Grill: 22/23 coding + R5 6/6 zero-loop, IDENTICAL across 2 of 3 reps (same
53 pocket-35b
pocket-35b-a3b-qwen3.5-256k		 llama.cpp Promoted — 256k 99 22.0/23 — — — 6.0/6 0.0/6 0 FINAL-Bench/POCKET-35B-GGUF:Q4_K_M (19.71GiB) — PROMOTED to driver 2026-09-14 (trial 2026-07-28). Fastest clean driver on the box: 22/23 coding, R5 6/6, 0 loops, ~102 t/s, and 4-for-4 clean R5 (24/24 tasks, zero loops in
54 qwen3-instruct
qwen3-30b-a3b-instruct-2507-131k		 vLLM Kept / trial — 128k 97 21.0/23 — — — 4.0/6 0.0/6 0 Qwen3-30B-A3B-Instruct-2507 (Alibaba, the July-2025 "2507" NON-thinking refresh of the Qwen3-30B-A3B MoE), served as a vLLM AWQ via the Sophia-AI W4A16 build. HEAD-TO-HEAD sibling of qwen3-thinking above: SAME base, SAME
55 kat-coder-fast
kat-coder-mtp-gguf, katmtp		 llama.cpp Active 35.5B 256k 95 20.2/23 — — — 5.5/6 0.5/6 0 KAT-Coder-V2.5-Dev-MTP, UD-Q4_K_XL GGUF (21.29 GiB) — the SAME base model as `kat-coder` above, but a different build, and THEY ARE NOT INTERCHANGEABLE.
56 euclide38
Euclide-Qwen38		 SGLang Kept / trial — 128k 89 21.0/23 19m05s — — 5.3/6 0.0/6 0 TRIAL 2026-10-13, GRILLED 3x 2026-10-13: 21/21/21 = 21.0/23, R5 5/6/5 = 5.33/6 with ZERO loops — TEXT-ONLY, nerkyor Qwen3.8-27B-Coder390 NVFP4 W4A4 + its DFlash2-FP8 drafter on SGLang 0.5.21 TP=2, grill-task median (this
57 kimi-distill
qwen3.6-35b-a3b-kimi-k2.6-distill-256k		 llama.cpp Promoted 34.7B 256k 89 21.2/23 — 7/23 partial — 6.0/6 0.0/6 0 PROMOTED to driver 2026-09-23. 21/23 coding, R5 6/6 ZERO loops / 0% redundant (verified solo + 2 concurrent in [[kimi-distill-x2-trial]]), ~100-106 t/s, 256k needle-verified (6/6 + 7/7 at 241k/243k). The coder-agentic ru
58 glm-flash-ck
glm-4.7-flash-awq-cyankiwi		 vLLM Kept / trial — 151k 88 22.0/23 — — — 6.0/6 0.0/6 0 CONTROLLED QUANTIZER A/B against `glm-flash-awq` — TRIAL 2026-08-31. SAME base model (zai-org/GLM-4.7-Flash), DIFFERENT quantizer: glm-flash-awq = QuantTrio/GLM-4.7-Flash-AWQ 18.41 GiB, 145k dl glm-flash-ck = c
59 gpt-oss-20b
gpt-oss-20b-mxfp4-4060ti-131k		 llama.cpp Promoted 20.9B 128k 88 22.3/23 — 66/134 2h55m — — 0 OUTSTANDING ONE-SHOT CODER, POOR AGENT — grilled 2026-08-29 (bench/grill-gpt-oss-20b-20260829-1046): coding 22/23, realcase 7/7, KLayout+ref 7/8. But R5 run 3x gave complete 4/6, 3/6, 3/6 with ONE GENUINE LOOP (search_
60 mellum llama.cpp Active 12.1B 128k 84 22.0/23 — — — 6.0/6 0.0/6 0 ON-DEMAND SWAP ENTRY. Was an always-on PEER on :9161 backed by the separate mellum-driver systemd unit (never started/stopped/evicted by llama-swap). Moved into the swap pool 2026-08-23: the driver owned the 4060 Ti outr
61 glm-flash-sglang
glm-4.7-flash-sglang-tp2		 SGLang Active — 128k — 20.0/23 — — — 6.0/6 0.0/6 0 FIRST SGLANG ENTRY — a FOURTH backend after llama.cpp, vLLM and TabbyAPI/EXL3. Wired like the vLLM and Tabby entries: llama-swap owns the lifecycle, so selecting this EVICTS the resident model instead of fighting it for
62 kat-coder
kat-coder-v2.5-dev-mtp-exl3-5bpw		 TabbyAPI Active — 252k 83 19.0/23 — — — 6.0/6 0.0/6 0 THE FIRST Qwen3.5-35B-A3B DERIVATIVE TO CLEAR `interrupt_replan`. That task is THE agentic discriminator here ([[grill-round5-agentic-loops]]) and it has killed every other fine-tune of this base: apodex looped 3/3 (RE
63 glm-flash-awq vLLM Active — 160k 82 20.0/23 — — — 6.0/6 0.0/6 0 THE EVERYDAY DRIVER. 26-27/29 coding over 2 repeats, R5 6/6 then 5/6, 82 tok/s, and 13/16 KLayout WITH an API reference despite being TEXT-ONLY (Glm4MoeLiteForCausalLM, zero vision tensors). Its output length ignores the
64 north-mini
north-mini-code-1.0-cohere2moe-488k		 llama.cpp Kept / trial 30.5B 488k 80 22.0/23 — — — 4.0/6 1.0/6 0 unsloth/North-Mini-Code-1.0-GGUF:UD-Q4_K_XL (17.93GiB) — TRIAL 2026-07-28. CohereLabs North-Mini-Code-1.0. ARCH: cohere2moe (Cohere2MoeForCausalLM) — SUPPORTED in this build: LLM_ARCH_COHERE2MOE is registered in llama.cp
65 qnext-fast
qwen3next-thinking, qwen3-thinking-80b		 llama.cpp Promoted 79.7B 256k 73 20.0/23 — — — 5.0/6 0.0/6 0 RENAMED 2026-09-26 from qwen3next-thinking (owner): the 2-bit, fully-in-VRAM twin of ${qnext} -- kept for 262k context and ~73 t/s. Old ids kept as aliases, so every existing client config still works. Qwen3-Next-80B-A3B
66 qnext
qwen3next-thinking-q3, qwen3-next-80b-thinking-q3		 llama.cpp Promoted 79.7B 128k 57 22.0/23 — — — 6.0/6 0.0/6 0 RENAMED 2026-09-26 from qwen3next-thinking-q3 (owner: short name); old ids kept as aliases. PROMOTED 2026-09-26 after a 3-rep grill -- see the verdict block below. TRIAL 2026-09-25 -- 3-bit upgrade of ${qwen3next-thinkin
67 coder-agentic
qwen3-coder-next-80b-256k		 llama.cpp Kept / trial 79.7B 256k 56 22.0/23 — — — 5.5/6 0.5/6 0 LONG-CONTEXT / agentic coder (replaced qwen3-coder-30b-q4-256k). Qwen3-Coder-Next 80B-A3B (3B active) on Qwen3-Next hybrid attention, purpose-trained for agentic coding (better SWE-bench Pro / long-horizon). UD-IQ2_M qua
68 darwin-35b-solo
darwin-35b-solo-262k, darwin-35b-5060ti		 llama.cpp Active 34.7B 256k 53 — — — — — — 0 SINGLE-CARD --n-cpu-moe variant of darwin-35b on the 5060 Ti, built 2026-09-30 so one card holds it alone and it runs alongside the `one-card` twins on the 4060 Ti (qwen38-gsq2, tommy2) — the dual-GPU `darwin-35b` entry
69 qwen38-gsq1
qwen3.8-27b-gsq-rco-5060ti, gsq-rco-5060ti		 llama.cpp Active 27.3B 80k 51 — — — — — — 0 SINGLE-CARD, TEXT-ONLY split of qwen38-gsq on the 5060 Ti, built 2026-09-30 so the same weights can serve while the 4060 Ti runs something else — the dual-GPU entry above needs both cards.
70 coder-agentic-q3
qwen3-coder-next-80b-q3		 llama.cpp Kept / trial 79.7B 128k 50 21.0/23 — — — 6.0/6 0.0/6 0 THIRD rung of the Qwen3-Coder-Next 80B-A3B quality ladder, added 2026-08-24. All three entries are the SAME base weights, kept side by side on purpose: coder-agentic UD-IQ2_XXS 21.71 GiB + q4_0 KV @262k (~72 t/s, 22
71 kolibri-1
kolibri-1-78b-a3b-256k		 llama.cpp Kept / trial 78.1B 256k 47 — — — — — — 0 Hob-forge/Kolibri-1-GGUF:Q4_K_M (44.2 GiB) — TRIAL 2026-10-06. Aleph Alpha Kolibri-1, arch `kolibri1`: 78B MoE, 3.46B active, 384 experts top-6 + 1 shared, 50 layers, 4:1 sliding(513)/full attention, 262144 ctx. NEEDS th
72 qwen38-gsq2
qwen3.8-27b-gsq-rco-4060ti, gsq-rco-4060ti		 llama.cpp Active 27.3B 80k 39 — — — — — — 0 IDENTICAL to qwen38-gsq1 above except for the device pin, so the pair can fill whichever card is free. Same weights file, same patched template, same sampler, same q8_0 KV, same MTP, same 81920 context — the 4060 Ti has
73 k2-horizon
k2-horizon-mova-36b-a4b, mova-36b		 llama.cpp Active 37.4B 128k 35 19.7/23 — — — 4.9/6 0.3/6 0 THE ONLY ENTRY ON ${ifm_llama_bin} — a SECOND llama.cpp fork build. IFM/MBZUAI K2-Horizon-MoVA-36B-A4B. Mainline llama.cpp CANNOT load this: `unknown model architecture: 'k2-horizon'`. Upstream master has no such arch
74 glm53-coder
glm-5.3-flash-reap144-coder-iq2s-131k, glm53-reap		 llama.cpp Active 161.1B 128k 8 — — — — — — 0 ADDED 2026-10-19, LOAD-TESTED, NOT GRILLED — lapadatbogdan/GLM-5.3-Flash-REAP144-Coder-GGUF IQ2_S (2 parts, 51.0 GiB): GLM-5.3-Flash 321B-A18B pruned to 144/288 experts per layer on Python agentic sessions (161B-A17B). C
75 qwen38-strata-coolbox llama.cpp Active — 256k — — — — — — — 0 Strata engine from the monorepo submodule (coolbox/strata, pinned v0.1.40), stock strata-iq2_xs.json (no layer split / trim-stage-weights, unlike qwen38-strata-fast). Same weights, pack and flags as the 2026-10-20 standa
76 strata-iq3 llama.cpp Active — 128k — — — — — — — 0 Strata IQ3_S (GSQ-RCO, same base as qwen38-strata-plus) from the coolbox/strata submodule, stock strata-iq3_s.json as run on :8080 on 2026-10-21 (131072 ctx, --kv-resident not set, --pcie-frac 0.20, --spec-min-p 0.70). -
77 agnes
agnes-3.0-flash, agnes-flash-256k		 llama.cpp Kept / trial 32.6B 256k — — — — — — — 0 Agnes-3.0-Flash (Agnes-AI, Apache-2.0) — the open-weight Preview checkpoint, via quimmedes/Agnes-3.0-Flash-XYZ-GGUF. A hybrid-attention (3 delta/linear : 1 global) reasoning model: 73 blocks, ctx 262144, 24 heads / 4 KV
— omni vLLM Not servable — 64k 132 — — — — — — yes THE ONLY MODEL ON THIS BOX THAT CAN HEAR. Qwen3-Omni-30B-A3B AWQ, 25.7 GiB, the largest model served here (15.2 of 16.3 GiB per card). Verified 2026-08-10: espeak-ng speech in -> 8/8 words back verbatim. vLLM reports Sup
— gpt-oss-120b
gpt-oss-120b-mxfp4-cpumoe-131k		 llama.cpp Not servable 116.8B 128k 21 — — 13/26 partial — — — 0 Free VRAM after the long prompt: 944 / 1170 MiB. ncmoe 23 left card 1 at 248 MiB for +0.5 t/s — keep 24 (cuBLAS died at 466 MiB in Aug). Decode is CPU-bound (6 cores ~570%, GPUs ~9%); no flag moves it. --swa-full is MAND

Not servable (2): omni (cyankiwi/Qwen3-Omni-30B-A3B-Instruct-AWQ-4bit), gpt-oss-120b (cause unresolved — the 2026-09-28 failure was measured on a config the abb6c751 edit had corrupted, not the one the entry now holds). Each is enabled in llama-swap.yaml but cannot serve a request. The ones whose cause is an HF repo had their weights moved off to /mnt/parked/hf-cache/ and the ~/.cache/huggingface/hub/ entries that point there are now dangling symlinks, so the backend dies before it binds a port and every request 500s — to restore one, copy its snapshot back to fast storage (/mnt/ssk500/models/) and repoint the symlink. The rest were declared not servable by hand: the entry is unchanged and its weights are present, but it cannot serve a request here, for the reason named in parentheses — the measurements behind that call are in the entry's own comment block in llama-swap.yaml. They are not ranked, and any tok/s or grill figure shown against one is a real measurement kept for the record, not a current claim.

FreeToken models — the peer slot on :9162

Three models served by ft (FreeToken) from systemd units rather than from llama-swap.yaml entries, which is why they have no row in the table above — the yaml knows them only as peer names. All three bind :9162 on the RTX 5060 Ti and conflict with each other, so only one can serve at a time; all three are installed but disabled, and started on demand. The score is the same 23-task coding suite the table above uses, taken from custom battery runs under bench/grill-*/ that no other loader reads. The tok/s is the documented decode rate for the config each unit serves now — not the rate measured during its grill, which for gemma4-ft was a 32k/offload run rather than the 131k/hybrid it serves.

Model Params Context MoE tok/s Grill (/23) R5 complete R5 looped Verdict
qwen36-ft 35B 256k — 47 21.0/23 6.0/6 0.0/6 Thinking MoE at the full 262k — the box's only genuine long-context endpoint. 21/23 with no R5 loops, ~2x the tokens of gemma4-ft per answer.
gemma4-ft 26B 128k — 27 22.3/23 5.7/6 0.3/6 Non-thinking and token-efficient: 22-23/23 across three reps, real-case 7/7 in all three, one R5 loop in three runs. Office is its weak suite.
gpt-oss-20b-ft 20B 128k — 60 22.0/23 3.3/6 0.3/6 Fastest decode on the box and an outstanding one-shot coder, but a POOR AGENT — R5 3-4/6 with a genuine loop; never wire it to tool loops.

Scores from: qwen36-ft from bench/grill-qwen36-ft-20260823-1553; gemma4-ft from bench/grill-gemma4-ft-20260823-1800, bench/grill-gemma4-ft-20260823-1821; gpt-oss-20b-ft from bench/grill-gpt-oss-20b-20260829-1046. Rounds 1-4 of each battery are the canonical suite (r1=8, r2=5, r3=5, r4=5 = 23), so these totals mean what the Grill column above means; a multi-rep battery is shown as its mean with the range in the tooltip, never as its best rep. Real-case gate, KLayout+API-ref and the office suites were run in the same batteries and are not shown here. The tok/s figures are decode rates under load, warm, on the card at PCI 01:00, and are context-dependent — do not compare them across different window sizes. The verdicts are curated from auto-memory/, which is where the evidence for each one lives.

Runs alongside FreeToken — the one-card entries on the 5060 Ti @ 02:00

The 6 entries below pin CUDA_VISIBLE_DEVICES=1 with CUDA_DEVICE_ORDER=PCI_BUS_ID, which is the RTX 5060 Ti @ 02:00. Every FreeToken driver pins CUDA_VISIBLE_DEVICES=0 — the 5060 Ti @ 01:00 — so these do not contend with it for VRAM and stay usable while a FreeToken model is up. Everything else on the box needs the 5060 Ti @ 01:00: an entry pinned to no card lands on it by default, any --tensor-split, -sm tensor or TP=2 entry spans both, and the entries that do pin CUDA_VISIBLE_DEVICES=0 are sitting on the driver's own card — see the section below. None of those can load while the driver is up: the first kinds for want of a card, the last because they compete with the driver for the 5060 Ti @ 01:00's VRAM. All 6 are GPU-resident, so none of them spills experts into the ~13 GB of host RAM the driver's pinned expert banks leave free.

The catch: these are a swap pool, not 6 concurrent servers — llama-swap keeps one of them loaded at a time and evicts the others. So each coexists with FreeToken, but not with the rest of this table. The two one-card twins are the exception: qwen38-gsq2 and tommy2 also sit in the one-card group described in the section below, so one of them can stay up while a member of that group holds the 5060 Ti @ 01:00. Their order and figures are carried over from the ranked table above, which is a ranking of the whole box; this section is a filter on it, not a second opinion on it.

Model Params Context Vision tok/s Grill (/23) Grill wall R5 complete R5 looped Verdict
tommy2
thomson-1.0-small-iq3xxs-4060ti-131k		 35B 128k yes 87 21.0/23 — 6.0/6 0.0/6 SINGLE-CARD CLONE of tommy on the 4060 Ti — runs IN PARALLEL with the FreeToken driver on the 5060 Ti (gemma4-ft / qwen36-ft). Same model (thomsonreuters/Thomson-1.0-Small, Qwen3.6-35B-A3B) + same patched family template
qwen3vl-8b 8B 128k yes 49 21.0/23 — 4.0/6 0.0/6 Qwen3-VL-8B-Instruct (Q4_K_M, 4.7 GB + 1.1 GB mmproj). Added 2026-08-23 as the smallest capable VISION model for co-residency — the 4060 Ti swap pool (mellum, gemma12, bonsai, glm-ocr) has no vision model of its own; thi
bonsai
ternary-bonsai-27b-196k		 27B 192k yes 35 19.5/23 — 6.0/6 0.0/6 THINKING MODEL: it spent 1,807 tokens answering "Blue" to a one-word vision question and ~1,300 tokens on a simple coding task. Raw decode is ~37 tok/s, close to gemma12's 42, but time-to-answer is ~4x longer. Budget acc
gpt-oss-20b
gpt-oss-20b-mxfp4-4060ti-131k		 21B 128k 0 88 22.3/23 — — — OUTSTANDING ONE-SHOT CODER, POOR AGENT — grilled 2026-08-29 (bench/grill-gpt-oss-20b-20260829-1046): coding 22/23, realcase 7/7, KLayout+ref 7/8. But R5 run 3x gave complete 4/6, 3/6, 3/6 with ONE GENUINE LOOP (search_
mellum 12B 128k 0 84 22.0/23 — 6.0/6 0.0/6 ON-DEMAND SWAP ENTRY. Was an always-on PEER on :9161 backed by the separate mellum-driver systemd unit (never started/stopped/evicted by llama-swap). Moved into the swap pool 2026-08-23: the driver owned the 4060 Ti outr
qwen38-gsq2
qwen3.8-27b-gsq-rco-4060ti, gsq-rco-4060ti		 27B 80k 0 39 — — — — IDENTICAL to qwen38-gsq1 above except for the device pin, so the pair can fill whichever card is free. Same weights file, same patched template, same sampler, same q8_0 KV, same MTP, same 81920 context — the 4060 Ti has

Members here: tommy2, qwen3vl-8b, bonsai, gpt-oss-20b, mellum, qwen38-gsq2. The selection is read from each entry's own cmd, so it follows the yaml rather than a list kept here. glm-ocr (0.9B OCR) is pinned to the 5060 Ti @ 02:00 the same way and also stays usable, but it is not a chat or coding model and so is excluded from the ranked table. gpt-oss-20b here is the llama.cpp entry on the 5060 Ti @ 02:00, not the gpt-oss-20b-ft FreeToken model in the section above — same base model, different deployment, separate grills. One caution: a FreeToken driver's hybrid MoE execution uses all six physical cores while decoding, so a co-resident model's own CPU-side work (prefill, sampling) competes with it. (The Xid 31 faults once noted here were seen on the RTX 4060 Ti that sat in this slot until 2026-10-09; that card is out of the box.) Both are recorded in auto-memory/.

The one-card entries on the 5060 Ti @ 01:00 — these take the FreeToken card

The 3 entries below pin CUDA_VISIBLE_DEVICES=0 with CUDA_DEVICE_ORDER=PCI_BUS_ID, which is the RTX 5060 Ti @ 01:00 — the card every FreeToken driver holds. So unlike the section above, this is not a free-card list: none of these can be up at the same time as a FreeToken model. A driver is a peer, which llama-swap proxies and never swaps, so nothing gives way to make room — the two simply do not both fit on the 5060 Ti @ 01:00. Their purpose is the mirror image — each holds a model on one card so the other card stays free. For darwin-35b-solo and qwen38-gsq1 that means taking a model the box would otherwise place on both cards and confining it to one; for tommy1 it means a second copy of a model that was already single-card, so the tommy pair can sit on whichever card is idle. The price is speed: measured 2026-09-30, darwin-35b-solo runs at ~53 t/s against the two-card entry's ~110-114, and qwen38-gsq1 at ~51 t/s. darwin-35b-solo also carries --n-cpu-moe, so its routed experts live in host RAM rather than on the card; that is what buys it a full 262144 context on one GPU, and host RAM is the resource it shares with the rest of the box.

Two catches — and the first one has just changed. The stock rule is that llama-swap keeps one managed model loaded at a time and evicts the rest, so a single-card pair does not become two concurrent servers merely by existing. Since 2026-09-30 it does: llama-swap.yaml carries a routing.router.settings.groups block whose single group, one-card, holds exactly the five twins built that day — qwen38-gsq1, darwin-35b-solo and tommy1 on card 01:00, qwen38-gsq2 and tommy2 on card 02:00. Members of a group with swap: false do not evict each other, so one from each card runs at once; measured 2026-09-30, qwen38-gsq1 and tommy2 answered together in 3.0 s and 5.1 s with both cards held. The group is kept narrow on purpose, and exclusive: true is what lets it be narrow safely: the ordinary swap survives in both directions — a request for a two-card model still unloads these, and a request for one of these still unloads a two-card model (both directions measured, same day). Three of the five share the 5060 Ti @ 01:00, and swap: false is per group rather than per card, so asking for two of those does not swap either: the second fails to allocate and the first stays up — unload it, or ask for a model on the other card. The older card-02:00 entries stay out of the group for the same reason (five rows in the section above, plus glm-ocr): they are models the box swaps between, and joining would cost them that swap. (The FreeToken models are the other exception, and only because they are declared under peers: — they are external systemd units llama-swap proxies rather than manages, so it never swaps them out, and peers sit outside the group system altogether.) Second, the card is not the only budget. Anything here that keeps experts in host RAM draws on a box-wide pool — whatever else the machine is serving, on either card, comes out of the same ~31 GiB — so read the spill note above before running one of these next to a second model. Figures and order are carried over from the ranked table above, which is a ranking of the whole box.

Model Params Context Vision tok/s Grill (/23) Grill wall R5 complete R5 looped Verdict
tommy1
thomson-1.0-small-iq2m-5060ti-131k		 35B 128k yes — — — — — SINGLE-CARD CLONE of tommy2 on the 5060 Ti, added 2026-09-30 — the exact mirror of the entry above, so the tommy pair can occupy BOTH cards the same way qwen38-gsq1/gsq2 do. Byte-identical cmd except CUDA_VISIBLE_DEVICES
darwin-35b-solo
darwin-35b-solo-262k, darwin-35b-5060ti		 35B 256k 0 53 — — — — SINGLE-CARD --n-cpu-moe variant of darwin-35b on the 5060 Ti, built 2026-09-30 so one card holds it alone and it runs alongside the `one-card` twins on the 4060 Ti (qwen38-gsq2, tommy2) — the dual-GPU `darwin-35b` entry
qwen38-gsq1
qwen3.8-27b-gsq-rco-5060ti, gsq-rco-5060ti		 27B 80k 0 51 — — — — SINGLE-CARD, TEXT-ONLY split of qwen38-gsq on the 5060 Ti, built 2026-09-30 so the same weights can serve while the 4060 Ti runs something else — the dual-GPU entry above needs both cards.

Members here: tommy1, darwin-35b-solo, qwen38-gsq1. The selection is read from each entry's own cmd — an entry moves between this section and the one above by changing its device pin, so neither list is maintained by hand. Every row here is also a row of the ranked table above.

Split entries — one model on both GPUs, two requests at once

What a split entry is. A normal entry answers one request at a time: a second request waits until the first one is finished. A *-split entry loads one copy of the model across both GPUs and opens two slots (--parallel 2), so two requests are answered at the same time — for example two coding agents, or an agent and a chat. Each one is the same weights as its single entry elsewhere on this page (tommy, darwin, swift15, pocket-vision, fable-turbo).

The trade-off is context per request. The context budget (-c) is shared by the two slots, so each request gets half of it: Total context below is what the server allocates, Per request is what one conversation can actually use. In exchange the whole model stays on the GPUs (nothing spilled to the CPU), so prompts are read several times faster than on a one-card entry that has to push expert layers through the CPU. Two streams at once share the GPUs' memory bandwidth, so each runs roughly 1.3-1.5x slower than alone, never 2x: the gain is total throughput, not the speed of one answer.

When to use which. Use a split entry when two clients work at the same time; use the single entry when one request needs the largest window or the fastest single answer. All five are built on Qwen models (Qwen3.5/3.6-35B-A3B MoE or Qwen3.8-27B dense), and all run inside llama-swap. The measured table after this one has the full grill details.

Entry Weights How it is spread over the 2 GPUs Total context Slots Per request KV cache (K / V) Images Grill (/23) Vision (/52) Grill wall
tommy-split
thomson-1.0-small-qwen35-x2, tommy-x2		 thomsonreuters_Thomson-1.0-Small-Q4_K_M.gguf tensor parallel (-sm tensor) 448k 2 224k q8_0 / q8_0 yes 21.5/23 50/52 31m40s
darwin-split
darwin-35b-vision-x2, darwin-vision-x2		 FINAL-Bench_Darwin-35B-A3B-Opus-Q4_K_M.gguf layers split 47 / 53 384k 2 192k q8_0 / q8_0 yes 21.2/23 52/52 17m22s
swift-split
swift-1.5-qwen3.8-27b-256k-x2, swift15-x2		 Swift-1.5-Qwen3.8-27B-Q4_K_M.gguf tensor parallel (-sm tensor) 320k 2 160k q4_0 / q4_0 yes 22.3/23 51/52 —
pocket-split
pocket-35b-vision-x2, pocket-vision-x2		 POCKET-35B-Q4_K_M.gguf layers split 49 / 51 256k 2 128k q8_0 / q8_0 yes 20.7/23 51/52 —
fable-turbo-split
qwen3.8-27b-turbo-fcf-735-882-x2, fable-turbo-x2		 Qwen3.8-27B-TurboFCFusion-735-882-MTP-Q4_K_M.gguf tensor parallel (-sm tensor) 256k 2 128k q4_0 / q4_0 yes — yes —

The *-split entries — measured details

These are a different operating mode from every other entry on this page. Every other entry here runs on one card, or spans two by cutting a single request between them. A *-split entry serves the whole model on both cards at once via --parallel 2, so two requests are genuinely in flight simultaneously — each on its own slot — instead of one waiting for the other to finish. The second card is not a spare: it is what stops prefill being the bottleneck. A one-card entry buys its window by spilling expert layers to the CPU (--n-cpu-moe), and every prefill micro-batch then has to push those layers through the CPU; keeping the experts on the cards measured 3728 t/s prefill against 405 for the same model, so a 42k-token re-prefill costs about 11 s instead of about 109 s. That gap is invisible in a /23 column and very visible in use.

The price is window per slot, because -c is the total KV budget and --parallel 2 halves it. darwin-split and tommy-split both doubled -c to pay for their second slot; pocket-split instead left -c 262144 alone and took the one-flag change, so the same VRAM now buys 131072 tokens per slot rather than 262144 in one. Concurrency here is a throughput win, not a latency one: two simultaneous streams share memory bandwidth and cost each about 1.3-1.5x, never 2x.

Measured 2026-09-30 — three serial reps each. The columns below read bench/grill-split-canonical-3x.sh: three independent reps of the full canonical grill (run_grill.sh = lm_eval + rounds 1-5) plus three reps of the 52-image vision suite, run one request at a time so each rep fills a single slot. Serially is the only way these entries can carry a published score at all — a GRILL_PARALLEL run against a --parallel 2 entry measures the model under contention, and both score loaders refuse those logs whole. The earlier contended campaign (bench/split-campaign-20260930-163836/) is superseded by this battery and is quoted nowhere on this page.

They are not ranked against each other, and the /23 column cannot separate them. Each entry runs the same weights as its single-card sibling elsewhere on this page, so the coding column largely re-measures a model already listed — and across three reps the three overlap almost completely: tommy-split 21.3/23 (reps 22/21/21), darwin-split and pocket-split both 20.7/23 (20/22/20 and 22/19/21). One task separates first from last, which is inside the spread. All three completed round 5 with zero loops. Read the Vision column with the same care: it is the best of each entry's three reps — darwin 52/52 (reps 52/51/50), pocket 51/52 (51/50/51), tommy 50/52 (50/50/50) — so a one-image gap there is a sample, not a ranking.

What a *-split entry actually buys is concurrency and prefill speed — keeping the experts on the cards is the 3728 t/s prefill against 405 quoted above — and what it costs is window per slot. The three differ far more on those axes than on coding: tommy-split holds the widest slot (229376 tokens, against darwin-split's 196608 and pocket-split's 131072) and decodes fastest, while pocket-split bought its second slot with window rather than VRAM — -c left at 262144, so the same VRAM now buys 2 x 131072 instead of 1 x 262144. Read each entry's own notes for its load-test and VRAM figures; all three are walled by expr_eval, the task every model on this box fails.

Decode rate and wall-clock are not the same number, and the gap is what picks the default. tommy-split decodes fastest — 114.0 against darwin-split's 95.4 t/s — but it emits far more tokens for the same work. Over the identical 30 timed tasks in each entry's three reps, output tokens summed from the logs: darwin-split 21k / 36k / 23k, pocket-split 23k / 50k / 32k, tommy-split 125k / 128k / 127k. tommy's spread is about ±2k — not a sampling artifact — and even darwin's worst rep against tommy's best is 3.4x. Over each entry's own decode rate, that is a 3-4x wall-clock loss (2.9x at its most generous to tommy, 5.0x at its least); on expr_eval, which every model here fails, darwin stops at 1773 output tokens in 18.8 s where tommy burns the 16000 cap in 145.3 s. Rounds 1-5 take darwin about 4.5 minutes, pocket about 5.5, tommy about 21. darwin-split is also the fastest to prefill (3728 t/s against tommy's 2460), and prefill — not decode — is what an agent pays on every turn as its conversation grows. That is why darwin-split, not the faster-decoding tommy-split, is the model claude-local starts by default; the one axis it concedes is window per slot — 196608 against tommy's 229376, the difference between serving this box's 223134-token deep-retrieval needle and rejecting it.

Model Params Context Vision tok/s Grill (/23) Grill wall R5 complete R5 looped Verdict
tommy-split
thomson-1.0-small-qwen35-x2, tommy-x2		 35B 224k 50/52 137 21.5/23 31m40s 5.8/6 0.0/6 The `darwin-split` shape applied to tommy: the SAME dual-card Q4_K_M entry with `--parallel 2` and a doubled `-c`, so one request gets both cards AND a spare slot. Added 2026-09-30, once darwin-split showed the shape was
darwin-split
darwin-35b-vision-x2, darwin-vision-x2		 35B 192k 52/52 110 21.2/23 17m22s 6.0/6 0.0/6 TWO SLOTS ON BOTH CARDS — the `darwin` entry above with `--parallel 2`. THE DAILY DRIVER since 2026-10-01 — re-instated as the `claude-local` default, and the reason `model-router` and the darwin-solo1/solo2 twins were
pocket-split
pocket-35b-vision-x2, pocket-vision-x2		 35B 128k 51/52 95 20.7/23 — 5.7/6 0.0/6 The `darwin-split`/`tommy-split` shape applied to pocket-vision, built 2026-09-30 on owner request. Same weights, same mmproj, same patched template: the ONLY change from pocket-vision is `--parallel 2`.
swift-split
swift-1.5-qwen3.8-27b-256k-x2, swift15-x2		 27B 160k 51/52 45 22.3/23 — 5.0/6 0.0/6 TWO SLOTS ON BOTH CARDS — `swift15` (Swift-1.5-Qwen3.8-27B) with `--parallel 2`. Wired 2026-10-01, modelled on `tommy-split`, which is the proof that the exact shape works on this box: `-sm tensor` + `--mmproj` + `--para
fable-turbo-split
qwen3.8-27b-turbo-fcf-735-882-x2, fable-turbo-x2		 27B 128k yes 37 — — — — THE FIT IS TIGHTER THAN swift-split's: this file is 18,498,573,856 B vs Swift 1.5's 17,442,399,936 B (+0.98 GiB, ~+0.5 GiB per card under tensor split). MEASURED 2026-10-07, cold loads against verified-empty cards: -c 32

Members here: tommy-split, darwin-split, pocket-split, swift-split, fable-turbo-split. Selected by reading each entry's own cmd — --parallel N with N of 2 or more and no CUDA_VISIBLE_DEVICES pin — so the list follows the yaml rather than a roster kept here. The pin test is what keeps it honest: glm-ocr runs --parallel 4 but is pinned to one card, and embed is an embedding model, so neither spans two cards and neither is listed. Context in the table is per slot, as it is everywhere on this page. Round-by-round logs: bench/grill-split-canonical-3x.sh writes bench/results-<entry>-<stamp>.log per grill rep and bench/results-vision-<entry>-rep<N>-<stamp>.log per vision rep — the second name matters, because the site reads vision runs from results-vision-*.log only, so a vision log named any other way is invisible here. Each entry's own measurements are recorded in auto-memory/.