laguna-xs-2-1

laguna-xs — RETIRED 2026-07-27, NO LONGER SERVABLE

Removed from the box 2026-07-27 ([[2026-07-27-model-cleanup]]): entry

deleted from llama-swap.yaml, 24GB Q5_K_M weights wiped from the HF cache.

Rationale: grill round 5 caught a genuine tool-call loop, and the headline

23/23 was run at a 32k token budget vs the standard 16k the rest of the

ladder used — not a clean apples-to-apples win. fable-fusion is now the

box's default agentic-coding pick. Everything below is history.

Original entry

poolside/Laguna-XS-2.1, served as laguna-xs (alias laguna-xs-2.1-224k),

bartowski Q5_K_M 24.0GB. 33B-total / 3B-active MoE, 40 layers, 256 experts top-8

+ 1 shared, hybrid global / sliding-window-512 attention. OpenMDW-1.1. Added

2026-07-26 in fbbba497. Purpose-built for agentic SWE coding, not chat.

GRILL 23/23 — perfect. Ties fable-fusion for the best coding result on this

box, and r4 agentic runs at 109.1 tok/step (982 tok, 1.8s median) — the leanest

among thinking models here (kimi-distill 159-166, glm-flash 162, bonsai 172,

tess-4-27b 238). CORRECTED 2026-07-27: it is NOT the leanest loop on the box.

[[devstral-small-2-24b]] is 37.4 tok/step and coder/gpt-oss-20b is 86.3 — both

non-thinking and leaner. The original claim came from comparing only against

kimi-distill and never checking the non-thinking coders. r1 8/8 including *both* lineage-flaky tasks (expr_eval — which

[[devstral-small-2-24b]] fails and [[gpt-oss-20b]] hits only 3-in-7 — and

median_two_sorted). Sustained 87–99 t/s decode.

The verdict is NOT ladder-comparable, and this is the important caveat. It ran

at GRILL_MAX_TOKENS=32000; every other entry on the ladder ran at the harness

default 16k. expr_eval alone emitted 17,026 output tokens and

median_two_sorted 13,178 — at 16k at least one r1 pass would have truncated

mid-think into a false FAIL. So part of the margin over devstral (21/23) and

kimi-distill (21/23) is *budget, not capability*. **Generalizes: the 16k default

may have been quietly under-measuring every verbose thinking model on this box.**

Re-run the ladder at 32k before ranking anything against this. Also, one run

never settles a flaky task — gpt-oss-20b's expr_eval record took 7 runs.

Laguna-S-2.1 (118B) is REJECTED — ~~unservable here, don't re-derive this~~.

**CORRECTED 2026-08-03 — see [[laguna-s-2-1-cpu-spill]]. The verdict stands but

the REASONING BELOW IS WRONG.** It *is* servable: UD-IQ2_XXS/IQ2_M (34.6-34.7GB)

run at 24 t/s with 7.4-8.1 GiB of MoE spilled to host RAM via -ncmoe, and

completed the full grill at ~21/23 + R5 6/6. Spill cost ~2.5x throughput, not

viability. It is rejected for cost/benefit — it occupies both GPUs *and*

7.4 GiB of RAM to score under [[pocket-35b]] (22/23 at 102 t/s, fully in VRAM).

Two specific errors in the original text: the "~19GB in use" was an idle ollama

pinning a retired model ([[ollama-keepalive-ram-pin]]), not a real floor; and

the smallest GGUF is now UD-IQ1_S at 31.4GB, not 33.8GB.

Original (wrong) reasoning: 118B-total / 8B-active. Smallest GGUF *in existence*

is unsloth UD-IQ1_S at 33.8GB, already over the 32GB pool before any KV;

official poolside GGUF floor is Q4_K_M at 68.2GB. --n-cpu-moe spill is not a

way out: 31GB host RAM with ~19GB in use, and 10-of-256 expert routing per token

means CPU-resident experts get hit constantly. XS is the only member of the

family that fits.

Four traps, each cost real time:

1. llama.cpp version floor. LLM_ARCH_LAGUNA landed upstream 2026-07-22

in 1f66c3ce1 ("Add support for Laguna XS.2 & M.1"). The box was on a 07-03

build and had to be rebuilt (now ff067f76d). Older binaries can't load it.

2. -rea on is mandatory. The GGUF's embedded template defaults

enable_thinking=false, but poolside's generation_config.json sets it true

and *all* their benchmarking used thinking mode. Without the flag you silently

bench a non-thinking model. --chat-template-kwargs is the deprecated spelling;

-rea/--reasoning on|off|auto is current.

3. Sampler is the vendor's, not this box's. temp 1.0 / top-k 20 / top-p

1.0 / min-p 0 — poolside's published benchmark settings, deliberately not

the usual 0.6/0.95/20 thinking profile. (Irrelevant to grill scores: the

harness sends its own temperature, so the yaml sampler only affects clients

that omit it.)

4. Context is capped by the COMPUTE buffer, not KV. At -c 262144 the weights

load fine and it dies later on allocating 2104.70 MiB on device N → "failed to

allocate compute pp buffers". KV is cheap on this arch (~1 layer in 4 is global,

the rest capped at a 512-token window, ~20KB/token at q8_0). **262144 is

unreachable at ANY split** — the buffer just migrates to whichever card is

tighter.

Split/context sweep (measured, free VRAM 4060Ti/5060Ti at 48,52):

131072 → 2707/1942 · 196608 → 1515/750 · 229376 → 1611/846 · 245760 → 1361/596 ·

249856 → 1297/532 · 253952 → 1235/470 · 262144 → FAIL. Non-monotonic —

229376 has a *fatter* margin than 196608 — because the split rounds to whole

layers. Shipped 229376 as the last rung at/above the box's tight-case precedent

(~1.1GB, qwythos-9b @680k); 253952 loads but at 470MB.

--tensor-split 48,52, not 50,50. Splits quantize to whole layers, so only

three states exist near the middle: 50,50≡49,51 (1879/2770 @131k),

48,52≡47,53 (2707/1942), 46,54≡44,56 (3309/1342). 48,52 has the best

*minimum* margin AND is the only split that loads at 245760+ (50,50 and 52,48

both fail). Remember A→CUDA0→4060 Ti ([[gpu-device-ordering]]).

Clean on the jinja guard — no non-first-system raise_exception; renders extra

system messages as their own <system> turn, so no chat-templates/ patch is

needed (a negative result worth not re-scanning, cf.

[[jinja-system-guard-tool-parser]]). Tool syntax is GLM-derived and custom

(<tool_call>name<arg_key>k</arg_key><arg_value>v</arg_value>); llama.cpp's

auto-parser carries an explicit [Patch: Laguna] case in

common/chat-diff-analyzer.cpp keyed on laguna_glm_thinking, which also

registers </assistant> as a literal stop string. Verified end-to-end: tools +

non-first system → HTTP 200, structured tool_calls.

TODO: poolside ships a DFlash draft head (poolside/Laguna-XS-2.1-DFlash,

GGUF at Lucebox/Laguna-XS-2.1-DFlash-GGUF) for speculative decoding — worth

trying the way MTP was added to [[tess-4-27b]]. Cost note: verbose, ~61k output

tokens for grill round 1 alone. See [[llama-swap-setup]], [[vram-spend-priority]].