← all models

nemotron-lightning

Kept / trial  active in llama-swap.yaml · aliases: nemotron-3.5-lightning-30b-a3b-262k

Grill run history

RunScoretok/stepR5Log
20260812-1836577/8results-nemotron-lightning-20260812-183657.log

llama-swap.yaml entry

  "nemotron-lightning":
    aliases: [nemotron-3.5-lightning-30b-a3b-262k]
    # bartowski/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-GGUF:Q5_K_M (25.11 GiB) —
    # TRIAL 2026-08-12, PARTIAL. ARCH: nemotron_h_moe (hybrid Mamba-2 / MoE /
    # attention), day-0 supported by the 2026-08-11 build 6e62ba538 —
    # LLM_ARCH_NEMOTRON_H_MOE is in llama-arch.cpp:93. No fork needed.
    #
    # NOT the REJECTED Nemotron-3-Nano ([[nemotron-3-nano-30b-a3b]]). Same vendor
    # and same 30B-A3B shape, different model, and it BEATS nano on nano's own
    # protocol (same port, same f16 KV arm):
    #   needle @262k   nano 3/5  ->  lightning 5/5   (nano FAILED depths 0.5, 0.9)
    #   R1             nano 6/8  ->  lightning 7/8
    # Do not triage it away on the earlier rejection.
    #
    # KV IS ALMOST FREE, hence f16 rather than q8_0. Read from the GGUF metadata
    # 2026-08-12: block_count 53, and attention.head_count_kv is nonzero on only
    # **7 blocks — [5, 12, 19, 26, 33, 42, 52]** — of which blk.52 is the unused
    # MTP/nextn head, so **6 attention layers are active**. Every other block is
    # Mamba/MoE, whose SSM+conv state is per-SEQUENCE not per-token (~48 MiB
    # total). At kv_heads 2 / head_dim 128 that is 6 KiB/token:
    #   f16  @ 262144 = 1.50 GiB      q8_0 @ 262144 = 0.80 GiB
    #   f16  @   1M   = 6.00 GiB      q8_0 @   1M   = 3.19 GiB
    # So this runs native context at FULL f16 precision — quality-first per
    # [[vram-spend-priority]] — and sidesteps the q8_0-V-cache rejection trap.
    # See [[kv-sizing-full-attention-layers]].
    #
    # -c 262144 IS A CHOICE, NOT THE MODEL'S CEILING. The GGUF declares
    # context_length = **1048576**, and unlike [[nemotron-3-nano-30b-a3b]] — whose
    # card claimed 1M while its GGUF exposed 262144 — that 1M is real in the
    # weights. q8_0 KV would make 1M fit (25.11 + 3.19 = 28.3 GiB). NOT RAISED
    # because it is UNVERIFIED and there is reason to doubt it: rope.freq_base is
    # 10000.0 with rope.scaling.finetuned = False, the Llama-2 default, where a
    # genuine 1M model normally carries a large rope base or YaRN. Retrieval is
    # verified only to 260,841 tokens. Test with needle.py at label ~600000
    # before trusting anything above 262144.
    #
    # Q5_K_M is the ceiling for weights: Q6_K is 31.95 GiB and Q8_0 32.60 GiB
    # against a 31.0 GiB usable pool. Q5_K_M at 25.11 leaves ~5.9 GiB.
    #
    # Expect these WARNINGS at load, both benign: "failed to fit params to free
    # device memory: n_gpu_layers already set by user to 99" (we set -ngl
    # deliberately), and a run of "model has unused tensor blk.52.nextn.*" — that
    # is the MTP/speculative head, which this build does not use.
    #
    # STATUS: verdict NOT reached. R1 (7/8, ~92-95 tok/s) and the needle are the
    # ONLY suites ever run; R2-R5, klayout(+ref), klvision, office and loop have
    # never executed. Its one R1 miss is `expr_eval`, the field-universal failure
    # that only thinkingcap has ever passed — so 7/8 is a clean R1 at field
    # standard. See auto-memory nemotron-35-lightning-30b.
    # STORAGE TIER: /mnt/ssk500, NOT the NVMe — moved 2026-08-13. It is a
    # single-role spare (440k-1M retrieval), not a driver, so it pays the slow-disk
    # penalty: ~330 MB/s measured on ssk500 vs ~1.08 GiB/s on the NVMe, i.e. ~76 s
    # to read 25.11 GiB instead of ~25 s. That +50 s is noise against the ~27 min
    # first-query prefill its actual role costs. Copy verified byte-identical
    # before the NVMe original was deleted (size, head-512MB cmp, tail-512MB
    # sha256 44fd8f528177340f). NVMe working set was coder-agentic +
    # fable-fusion only — STALE as of 2026-08-14, it is now those two plus
    # thinkingcap, gemma-awq and fable-711-gptq. See [[nvme-working-set]].
    cmd: |
      /usr/bin/env
      CUDA_DEVICE_ORDER=PCI_BUS_ID
      ${llama_bin}
      -m /mnt/ssk500/gguf/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-Q5_K_M.gguf
      --alias nemotron-lightning
      --jinja -ngl 99 -c 262144 -fa on
      --tensor-split 48,52
      -b 2048 -ub 512
      --host 127.0.0.1 --port 9121 --parallel 1
    proxy: http://127.0.0.1:9121