kimi-dev-72b

Kimi-Dev-72B (unsloth/Kimi-Dev-72B-GGUF:UD-IQ2_XXS) — REJECTED

Why rejected: Dense 72B model (Qwen2.5-72B base) at extreme quant, same problems as [[deepseek-r1-distill-llama-70b]].

Test Results (UD-IQ2_XXS, 25.7 GB weights)

• Context: Max 64K at q4_0 KV (128K OOMs). Only 412 MiB free at 64K.
• 128K: OOM (cudaMalloc failed: 5904 MiB allocation)
• 32K q8_0: Fits (1.2 GB free)
• Speed: 10.3-12 t/s (6-8x slower than MoE models)
• Grill: FAIL on first task — 16K token cap hit, 25+ min/task, SyntaxError from ◁think▷ tags
• Quality: 2³²=512 was correct (unlike DeepSeek IQ2_XXS which got 2.0)

Problems

1. Dense 72B at IQ2_XXS = extreme quant degradation (same class as rejected DeepSeek)

2. ~10 t/s (6-8x slower than MoE models at 60-85 t/s)

3. Max 64K context (vs 128-256K for MoE models)

4. Produces ◁think▷ tags that break grill code extraction

5. Extremely verbose: hits 16K token budget per task (25+ min/task on this hardware)

6. 412 MiB VRAM margin at 64K (no room for larger context or better KV)

Comparison: existing MoE models (Ornith-1.0-35b, Agents-A1-35b, Qwen3.6-35b) all provide better quality, speed, and context.

Removed from: not added to llama-swap.yaml or models.json