kimi-linear-48b

kimi-linear-48b — RETIRED 2026-07-27, NO LONGER SERVABLE

Removed from the box 2026-07-27 ([[2026-07-27-model-cleanup]]): entry

deleted from llama-swap.yaml, 22GB IQ3_M weights wiped from the HF cache.

Rationale: weakest algorithmic grill score on the box (18/23) plus a genuine

tool-call loop caught in grill round 5. Everything below is history.

Original entry (2026-07-16)

moonshotai/Kimi-Linear-48B-A3B-Instruct (mradermacher IQ3_M quant,

LLM_ARCH_KIMI_LINEAR — 7/27 layers full MLA attention, 20/27 linear KDA).

Root cause of the "upstream command exited prematurely" crash: the yaml's

--cache-type-v q8_0 fails llama_init_from_model — this arch's V head dim (72)

isn't a multiple of q8_0's block size (32), so it errors before any VRAM is

touched ("V cache type q8_0 with block size 32 does not divide n_embd_head_v=72").

K cache at q8_0 is fine (its head dim divides cleanly). Fix: --cache-type-v f16,

keep --cache-type-k q8_0. Already applied in llama-swap.yaml.

Found via a foreground repro exactly as [[llama-swap-setup]] describes: run the

model's exact cmd directly (not through llama-swap) so real stderr is visible —

llama-swap only logs its own "upstream command exited prematurely" WARN and

does not forward the child process's stderr/stdout to the journal.

VRAM at c=262144: ~13.7GB/13.4GB per GPU (~2.6-3GB headroom each on this

box's 16GB cards) — tighter than the yaml comment's original ~10GB-free

estimate, since f16 V costs more than the originally planned (broken) q8_0 V

would have.

Grill result (temp 0.2, same harness as [[glm-4.7-flash]]/reason): 18/23.

Round1 (algorithmic) 5/8, round2 (algorithmic) 3/5, round3 (tool-call/format)

5/5, round4 (agentic loop) 5/5. So: weak single-function algorithmic

correctness (~62%), but perfect single-turn tool-calling/format adherence and

perfect multi-step agentic loop behavior (declines to fabricate, recovers from

errors, chains tool calls correctly).

Speed: ~85-95 tok/s decode in short/medium tasks — fast for a 48B model,

consistent with only 3B active params + most layers being linear-attention.

Agentic loop (round4) was very snappy: 9.5s wall total across 5 tasks, 1.6s

median wall/task, only 74.7 tok/step — quite decisive/low-token compared to

reason's round4 (56.4s wall, 769 tok/step).

How to apply: don't recommend for algorithmic/coding-correctness work

(gpt-oss-20b or glm-flash are stronger there per [[gpt-oss-20b]] /

[[glm-4.7-flash]]). Reasonable candidate specifically for tool-calling-heavy or

agentic-loop-heavy workloads where speed/decisiveness matters more than

raw algorithmic correctness — but re-verify since 18/23 is below both

gpt-oss-20b (23/23) and glm-flash (20/23) on the overall grill.