nemotron-3-nano-30b-a3b

nemotron-nano — NVIDIA-Nemotron-3-Nano-30B-A3B Q5_K_M — REJECTED 2026-07-29

bartowski/nvidia_Nemotron-3-Nano-30B-A3B-GGUF:Q5_K_M, 24.35 GiB. Never added to

llama-swap.yaml — trialled standalone on port 9121 only. Entry + 25GB weights

deleted same day. Grill log bench/results-nemotron-nano-20260729-153724.log,

needle log bench/needle-nemotron-nano-20260729.log kept as evidence.

Arch nemotron_h_moe (52 layers: 23 Mamba / 23 expert / 6 attention,

128 experts / 6 per token, 2 kv-heads, head_dim 128). First NVIDIA lineage and

first hybrid-Mamba model tried on this box.

Why rejected: dominated, not bad

| | 23-task | R5 | loops | redundant | decode |

|---|---|---|---|---|---|

| pocket-35b | 22/23 | 6/6 | 0 | 0.0% | 102 t/s |

| nemotron-nano | 20/23 | 5/6 | 0 | 7.7% | ~90 t/s |

R1 6/8, R2 4/5, R3 5/5, R4 5/5. Zero REQ-ERRs. This is a **genuinely good agentic

model** — a clean R5 with 0 loops beats [[north-mini-code]]'s 4/6-with-a-loop —

but pocket-35b beats it on every axis, so it earns no slot. The only unique

claim was NVIDIA-lineage decorrelation (the fleet is nearly all Qwen3.5/3.6), and

that isn't worth 25GB.

The advertised 1M context is not real

GGUF metadata says context_length: 1048576; the HF config.json says

max_position_embeddings: 262144. Trust the config. Three test runs were

burned probing 698k/831k before checking — those are ~3x out of spec and their

results are uninformative. Same class of unverified-context claim as writer's

never-tested 680k.

Needle at 262k (in spec), 5 depths, f16 KV: 3/5 — 0.1, 0.25, 0.75 PASS;

0.5, 0.9 FAIL. An earlier 3-depth q8_0 run gave 2/3 with 0.9 PASSING.

So retrieval past ~the first quarter is unreliable, and north-mini (3/3 at

440k) remains the long-context model.

Two reusable methodology findings

1. KV precision was NOT the cause — controlled f16-vs-q8_0 run at identical

ctx/split/batch: depth 0.5 failed both times with the *same* ct=1287. Do not

assume q8_0 KV explains a retrieval miss; test it, it's ~20 min.

2. A needle probe is a SAMPLE, not a measurement — depth 0.9 flipped

PASS→FAIL between two runs of the same weights at temperature 0. The same

caution [[round5-is-a-sample-not-a-measurement]] applies to R5 applies here:

do not read fine structure ("clean U-shape", "only the first 10% works") off a

single 3-point run. Repeat depths before claiming a curve.

Also: needle.py's TARGET argument is an *estimate* — this tokenizer inflated

it 1.66x (500000 requested -> 830840 actual). Pass target = wanted / 1.66.

bench/needle.py gained NEEDLE_MAX_TOKENS (default 2000, unchanged) this

session because reasoning models can spend the whole cap thinking and never state

the code, which the grader records as MISSING — a harness artifact. It was NOT

what caused these failures (a 12000 cap changed nothing), but the knob is right.

How to apply

A hybrid-Mamba MoE is servable and fast here, and the arch is not the problem —

literature says hybrids at ~7:1 Mamba:attention match transformers on NIAH, and

this one is ~3.8:1. If another nemotron_h/granitehybrid/jamba candidate

comes up, the shape is fine; check the config's real context window, and grill it

before believing any long-context claim. [[granite-4.0-h-small]] was the other

candidate triaged the same day and remains untried.