ernie-45-vl-28b-thinking

Trialled 2026-08-07 as the lineage-diversity probe: every VLM tried on this

box is Qwen-derived (fable-fusion, Bonsai, vision-coder, qwen3-vision,

qwen3-vl-thinking), and [[qwen36-27b-deep-retrieval-fails]] shows that family

shares failure modes. ERNIE is Baidu's, architecturally unrelated, so it can

tell us which failures are the family's and which are universal.

| Suite | ERNIE | qwen3-vision | fable-711-gptq | qwen3-vl-thinking |

|---|---|---|---|---|

| Coding total | 18/23 | — | 21/23 | 18/23 @16k |

| KLayout from memory | 0/8 | 1/8 | 0/8 | 0/8 |

| KLayout + API ref | 0/8 | 5/8 | 7/8 | 6/8 |

| Vision ground truth | 4/6 | 5/6 | 4/6 | 4/6 |

| Closed loop | 0/2 | 0/2 | 2/3 (unverified) | 0 real |

| Office | 2/5 | 0/3 | 2/5 | 3/5 |

| Throughput | 103 tok/s | ~140 | ~28 | ~140 |

REJECTED. Tool calling is clean (5/5, ernie45 parser) and perception is

fine, but it cannot write KLayout code at all and the closed loop never got

past codegen (TypeError: Can't match arguments, kdb.Rect does not exist).

**The finding worth keeping: an API reference is necessary but not

sufficient.** [[klayout-api-recall-vs-composition]] established that KLayout

failure is recall, not reasoning — hand over the signatures and models jump

+4 (qwen3-vision), +6 (qwen3-vl-thinking), +7 (fable-711-gptq). ERNIE gains

nothing: 0/8 → 0/8. It reads the same reference and still emits wrong

overloads and unit errors in BOTH directions at once — dbu_rect came out

1000x too large (area off by 10^6) while path_polygon came out 1000x too

small, and merge_stats never defined the required function name after

burning the full 16,000-token cap. So the reference only helps a model that

can already compose against a signature list; it does not install the skill.

Confirms the KLayout wall is NOT Qwen-specific. Four models across two

unrelated lineages now score 0–1/8 from memory. That kills the most

interesting alternative explanation and reinforces: never run KLayout work on

a local model without the klayout-classes skill loaded.

Serving it is painful — do not trust weights/TP for a VLM. vram_fit

predicted 9.15 GiB/GPU and a 304k-token pool from "18.3 GiB / 2". Reality:

"Model loading took 12.96 GiB memory" PER GPU

12.96 x 2 = 25.9 GiB against 18.3 GiB of weights, i.e. TP=2 replicates the

vision tower and embeddings instead of sharding them. At util 0.90 vLLM dies

with "No available memory for the cache blocks". Working config is

--gpu-memory-utilization 0.97 --max-num-seqs 2 --max-model-len 32768 for a

65,392-token pool — versus Qwen3-VL getting 131k in LESS memory on the same

two cards. **Read "Model loading took" from the startup log; never infer

per-GPU weights by dividing by TP.** Related: [[kv-sizing-full-attention-layers]].

Three separate blockers before it would even start, all now fixed in

vllm/trial-vlm-ernie45.sh:

1. --trust-remote-code — required, vLLM refuses the repo without it.

2. ImportError: decord — the remote code needs it; installed into

~/vllm-env with --no-deps (a transitive numpy bump would break vLLM).

3. OOM during profiling — it is a video model (hence decord) and vLLM

profiles peak memory against the worst-case multimodal input. Fixed with

--limit-mm-per-prompt '{"image": 1, "video": 0}'. Any video-capable VLM

on this box will need that cap.

WEIGHTS DELETED 2026-08-08 — 18.8 GiB reclaimed from

~/.cache/huggingface/hub. Re-downloading is ~5 min at the measured 59 MB/s,

so nothing here is lost that matters; the three startup blockers above and the

working config are preserved in vllm/trial-vlm-ernie45.sh, which was KEPT.

The verdict does not need re-testing: 0/8 KLayout WITH the API reference, in a

run where that same reference is worth +4..+7 to every Qwen model.