ornith-15-35b

ornith-15-35b-trial (ornith-15-35b-trial.md)

ornith-15-35b — Ornith-1.5-35B-A3B Q4_K_M, TRIAL 2026-08-21 (grilled, verdict below)

ornith-ai/Ornith-1.5-35B-A3B-GGUF:Q4_K_M (20.22 GiB) + mmproj-Ornith-1.5-35B-BF16

(0.84 GiB), on /mnt/ssk500/gguf/ (direct -m, the qwen38 pattern — not -hf,

so the cold load is timed with no download-during-load confound and no

offline dangling-refs gotcha [[llama-cpp-offline-hf-refs]]).

Arch qwen3_5_moe (Qwen3_5MoeForConditionalGeneration) — SAME family as

[[pocket-35b]] (Qwen3.5-35B-A3B) and [[qwen36-35b-awq-trial]] (Qwen3.6-35B-A3B).

Real sparse MoE (256 experts, 8/tok, ~3B active), 40 layers

(full_attention_interval: 4 = 10 full-attn, rest linear/DeltaNet), head_dim

256, kv_heads 2, 262144 ctx. Vision-capable (qwen3_5_moe_vision, depth 27)

— mmproj WIRED, unlike text-only pocket-35b. THINKING model (CoT →

reasoning_content). No MTP head.

Why this artifact / this path

Ornith-1.5 ships 3 scales; only the 35B-A3B fits this 2×16 GB box:

• 397B MoE (flagship): Q4_K_M 241 GB — no (even the [[laguna-s-2.1-cpu-spill]]

118B trick would be ~3.4× that weight → catastrophic prefill).

• 35B-A3B: fits. vLLM only via ulkaa/Ornith-1.5-35B-A3B-AWQ-INT4 (24.33 GiB,

0 downloads, unknown quantizer) — the official FP8 (~35 GB) doesn't fit

32 GB and the official NVFP4 is killed by the Ada card (SM 8.9, mixed-box

weakest-SM rule [[vram-spend-priority]]). AWQ on qwen3_5 arch has failed

systemically before ([[fable711-awq-self-quantize-todo]]: the (1+weight)

RMSNorm breaks AWQ smoothing). So vLLM is NOT the preferable path here — the

official GGUF (53k dl, the model's own org) is the higher-trust artifact.

Chose llama.cpp Q4_K_M despite the user's vLLM preference, and said why.

• 9B dense: fits trivially; less interesting (competes with small models,

not the KLayout use case).

The blog claims the 35B-A3B "outperforms Qwen3.6-35B across all coding/agentic

benchmarks." The grill tests that apples-to-apples vs the served qwen36-35b

(KLayout+ref 15/16, the box's co-leader) and pocket-35b (22/23, R5 6/6) on

IDENTICAL serving (same arch family, q8_0 KV / 256k / sampler; only the model

differs). Ornith 1.0 was served here as agent-mini (9B, 18/23 + R5 6/6,

per bench/MODEL-RANKING.md) + a 35B, both retired 2026-07-27.

Wiring (mirrors pocket-35b exactly — qwen3_5_moe is a proven drop-in)

llama-swap.yaml entry ornith-15-35b (port 9147), aliases:

[ornith-15-35b-a3b-256k]. cmd mirrors pocket-35b: ${llama_bin}, -m +

--mmproj (direct ssk500 paths), --jinja --chat-template-file

chat-templates/qwen35moe-nonfirst-system-256k.jinja, -ngl 99 -c 262144 -fa on,

--tensor-split 48,52, -b 2048 -ub 512, temp 0.6/top-p 0.95/top-k 20/min-p 0,

--cache-type-k q8_0 --cache-type-v q8_0, --parallel 1.

• The chat-template-file is LOAD-BEARING — the qwen3_5_moe family template

has the non-first-system guard; multi-turn system+tools 400s without the

patched copy, and a single-turn smoke test does NOT catch it

([[jinja-system-guard-tool-parser]]). The family-neutral file is

byte-identical to pocket-35b's/kat-coder's. NOT hit by the peg-native bug

([[llama-cpp-peg-native-tool-parser-500]]).

• q8_0 V-cache is accepted by qwen3_5_moe (pocket-35b proves it; despite

the CLAUDE.md "some archs reject q8_0 V" warning).

• Vision (mmproj) WORKS on llama.cpp for this arch — verified: described a

test image correctly ("black helicopter... Statue of Liberty"). This is the

FIRST qwen3_5_moe vision model served here (pocket-35b was text-only).

• models.json llama provider: ornith-15-35b + alias, input:["text",

"image"]. The input modality is NOT derived from --mmproj — it comes from

the VISION_IDS set in .claude/sync-models.sh ("a missing id silently

downgrades to text-only"). Both ids added to VISION_IDS, then

sync-models.sh run (regenerates the llama provider from the yaml + updates

both statusline copies). pi/prime inherit via the models.json symlink.

• Agent configs with their own files updated: ~/.kimi-code/config.toml

([models.ornith-15-35b], provider coolbox, capabilities=["image_in",

"thinking"]), ~/.omp/agent/models.yml, ~/.config/opencode/opencode.json

(with modalities + attachment: true like qwen38).

Load-test (all PASS, 2026-08-21)

Picked up via kill -HUP <MainPID> (no sudo) — re-reads llama-swap.yaml in

place; the resident vLLM TP=2 was torn down and **both workers reaped to 18 MiB

on both GPUs** by the [[qwen3-30b-a3b-thinking-vs-instruct]] orphan-reaper

wrapper (clean — no squat). Then:

• Cold load 97s, both GPUs balanced (4060Ti 12.9 GB / 5060Ti 14.5 GB,

headroom at q8_0 KV / 262144).

• Answered "OK" correctly (small 125-char reasoning_content, finish: stop).
• Multi-turn system+tools 200 OK — the guard case returns 200 with the

patched template (answered "hello.py", no spurious tool call).

• Vision 200 OK — mmproj describes an image correctly.

Grill (DONE 2026-08-21, 1 rep, 32 min — verdict: PARITY, not a win)

bench/grill-ornith-15-35b.sh ran all suites, GRILL_MAX_TOKENS=32000,

GRILL_TEMP=0.2, 1 rep, system python3 ([[klayout-office-grill-rounds]]).

Log: bench/grill-ornith-15-35b-20260821-1657/battery.log. Decode ~93 tok/s

(32000 tok / 344.8 s on realcase — pocket-35b's 102 tok/s class).

| suite | score | note |

|---|---|---|

| coding R1-4 | 21/23 | r1 6/8, r2-4 5/5 each |

| R5 agentic | 5/6 | clean — 0 loops, 0.0% redundant |

| klayout (no ref) | 0/8 | family baseline — runaways without ref |

| klayout +ref | 7/8 | min_spacing runaway @32000 — the ONLY true runaway |

| klvision | 5/6 | nested_rings FAIL (shape-detection miss) |

| closedloop (no ref) | 0/4 | both code arms fail → vision BLOCKED |

| closedloop +ref | 3/4 | ref closes the code loop; loop_ring:vision miss |

| office | 8/9 | bar_regions:vision miss |

| office +ref | 9/9 | perfect with ref |

| realcase | 7/7 PASS | cap-hit @32000/344.8s — verbose pass, NOT a runaway (all 7 sub-checks green) |

Runaway cleanliness: 1 true runaway (min_spacing) + 1 verbose-cap-hit-pass

(realcase, correct). qwen36-35b had 9 cap-hits — ornith is CLEANER here,

a point in its favor.

### The blog claim — NOT supported

Ornith-1.5's blog says the 35B-A3B "outperforms Qwen3.6-35B across all

coding/agentic benchmarks." It does not. It is parity, not a win — and it

is dominated on every individual axis by an incumbent:

• **vs pocket-35b (DIRECT comparison — same 23-task coding suite, same

grill_round*.py): coding 21/23 vs 22/23, R5 5/6 vs 6/6** — pocket-35b

ahead by 1 on both. pocket-35b is text-only (no vision/klayout/office), so

this only covers the shared coding+R5 axes, but pocket wins both.

• **vs qwen36-35b (compare by RATE, not raw — the grill was REFACTORED between

its 08-09 run and now: klayout/klvision/office/closedloop suites were roughly

HALVED, coding 46→23, R5 12→6; raw scores are not directly comparable):**

• coding: ornith 91.3% vs qwen36 89.1% — ≈ tie
• R5: ornith 83.3% vs qwen36 100% — qwen36 ahead (but R5 is high-variance,

n=1 — [[round5-is-a-sample-not-a-measurement]])

• klayout unaided: 0% vs 12.5% — both ~0
• klayout +ref: ornith 87.5% vs qwen36 93.75% — qwen36 ahead
• klvision: ornith 83.3% vs qwen36 75% — ornith ahead
• closedloop no-ref: ornith 0% vs qwen36 50% — qwen36 ahead (notable gap;

qwen36 was only the 2nd model EVER non-zero there)

• closedloop +ref: 75% vs 75% — TIE
• office: ornith 88.9% vs qwen36 83.3% — ornith ahead
• office +ref: 100% vs 100% — TIE
• realcase: ornith 7/7 — qwen36 was not graded on realcase in its trial

So ornith TIES on closedloop+ref and office+ref, BEATS qwen36 on klvision +

office(unaided), but LOSES on R5, klayout+ref, and closedloop-no-ref. Mixed, not

a sweep.

Deep retrieval — 256k is REAL: needle 3/3 @234k (2026-08-21)

The one open caveat on the verdict below was whether ornith's 256k window

actually retrieves or is generation-only like the qwen35 27B GGUF entries.

Tested after the 42,58 retune (model resident/warm), bench/needle.py 141000

0.1,0.5,0.9 NEEDLE_MAX_TOKENS=4000 via llama-swap :8090 (the qwen38 method):

pp=234,276 depth 0.1 PASS 142.1s in[content] ans='7Q-4412-ZX'

depth 0.5 PASS 144.8s in[content] ans='7Q-4412-ZX'

depth 0.9 PASS 145.0s in[content] ans='7Q-4412-ZX'

3/3 — code found in content at every depth. Per-depth times (~145 s each, no

~70x speedup) confirm COLD prefills, not a warm-cache replay

([[needle-warm-cache-invalidates-repeats]]). No depth-0.9 JSONDecodeError flake

(the qwen38 risk). The 42,58 split held through the full 234k prefill (4060Ti

12.8 GB / 5060Ti 14.6 GB, no OOM).

This narrows the cause, not just for ornith. [[qwen36-27b-deep-retrieval-fails]]

recorded fable-fusion / vision-coder / bonsai all 0/3 @222k on llama.cpp GGUF +

q8_0 KV + full_attention_interval 4, and called it architectural. ornith shares

ALL of that — llama.cpp, GGUF (Q4_K_M), q8_0 KV, full_attention_interval 4 (10

full-attn of 40) — yet retrieves at 234k. So the 0/3 @222k is specific to those

27B builds/weights, NOT to the llama.cpp GGUF q8_0 KV serving path or the

hybrid-attention design in general. (bonsai-awq already narrowed it by changing

quant+engine+KV together; ornith holds all three constant and varies only the

model — a cleaner isolation of "it's the build, not the path.")

Scope: one code at three depths, not a recall curve — the window is

addressable, not that reasoning over 234k is reliable. The 35B MoE siblings

pocket-35b and qwen36-35b were NEVER needle-tested, so this clears ornith and

the serving path, NOT the whole qwen3_5_moe family. Prefill is fast for a 234k

prompt (~145 s, ~3.7x qwen38's ~527 s at the same size — the sparse-MoE

prefill advantage), so the far end is more usable than qwen38's ~9-10 min,

though still not interactive.

### Verdict — KEEP as selectable TRIAL, do NOT promote

It does not win a quality role — every axis it contests, an incumbent already

wins: pocket-35b (coding+R5), qwen36-35b (klayout+ref, closedloop, the one-entry

vision+KLayout niche — qwen36 already replaced qwen3-vision AND

qwen3-vl-thinking), gemma-awq (coding+office, 44/46+18/18), qwen3next-thinking

(realcase+thinking champion, 7/7, already promoted). **But the differentiator

that keeps it is SPEED, not the capability combo.** qwen38 has vision+thinking

+256k too — but it is a DENSE 27B decoding at ~32 tok/s even with MTP,

while ornith is a sparse MoE (~3B active) at ~93 tok/s — 3× faster. Both

are thinking models (qwen38 even defaults reasoning_effort: xhigh), so tok/s

matters for both. **Wall-time on the SAME realcase task (both cap at 32000

tok): ornith 344.8 s → 7/7, qwen38 ~1048 s → 7/7 (and a 0/7 runaway in 3 reps)

— the 3× tok/s is a measured 3× wall-time to the same result.** For a thinking

model's long CoT, 32 tok/s is ~17 min/realcase vs ornith's ~6 min, so

**qwen38 does NOT substitute for ornith on the fast vision+thinking use

case.** qwen36-35b is fast+vision but has no thinking head; qwen3next-thinking

is fast+thinking but has no vision; qwen3-vl-thinking is the only other served

fast+vision+thinking model, and it carries documented quality red flags (false

geometry passes, runaways — see [[qwen3-vl-thinking-trial]]) where ornith passed

realcase 7/7 cleanly. So ornith's real, NON-redundant niche is **the fast

(MoE-class ~93 tok/s) vision+thinking+256k option with a clean realcase gate**

— and 256k retrieval is now VERIFIED (needle 3/3 @234k, above), so the window

is real, not generation-only like the qwen35 27B GGUF entries.

KEEP as selectable trial on that basis. (Verdict reasoning corrected

2026-08-21 after the AWQ trial underweighted speed and wrongly called it

redundant with qwen38.)

Caveats on the verdict:

• n=1. R5 and realcase are the two highest-variance axes

([[single-run-scores-are-unreliable]], [[round5-is-a-sample-not-a-measurement]]

— 6 identical R5 runs once spanned 4/6-6/6). A re-grill (2-3 reps) would only

matter if ornith were a borderline promotion candidate; it isn't (dominated on

each axis), so a re-grill is NOT auto-justified — left to the user.

• Vision shape-discrimination weakness: the 3 vision FAILs (nested_rings,

loop_ring:vision, bar_regions:vision) are all the same failure mode — losing

fine geometric detail (counting holes/rings/regions in shapes). The mmproj

describes broad scenes well (klvision 5/6) but loses fine geometry. First

qwen3_5_moe vision served, but not the box's vision pick (qwen36/gemma-awq

beat it).

• Serving differs from qwen36: ornith is Q4_K_M GGUF on llama.cpp; qwen36 was

AWQ on vLLM. The comparison is across serving stacks as well as weights.

### Why I keep it rather than remove

It's already wired into all 6 config surfaces and loads cleanly (97s cold, q8_0

KV, balanced GPUs). It costs nothing idle (llama-swap only loads on demand). It

is a legitimately usable vision+thinking 256k option for a user who wants the

qwen3_5_moe family with thinking (pocket-35b has no vision, qwen36 has no

thinking head). Demote-to-ssk500 is not warranted — it's not broken, just not

best. Leave as a selectable TRIAL.