← all models

tommy2

Active  active in llama-swap.yaml · aliases: thomson-1.0-small-iq3xxs-4060ti-131k

Grill run history

RunScoretok/stepR5Log
No results-*.log found for this id/alias.

Memory notes

tommy2-iq2m-grill (tommy2-iq2m-grill.md)

tommy2 = bartowski IQ2_M (12.07 GB) single-card clone of [[thomson-1.0-small-trial]]

(tommy, Q4_K_M 21.39 GB, 262k, both cards). llama-swap entry, 4060 Ti only,

131072, q4_0 KV, shared mmproj (vision works). Grilled 2026-08-29, 101 min,

artifacts bench/grill-tommy2-20260829-1527, script bench/grill-tommy2.sh.

**Two questions were under test: did 2-bit break the parent's clean agentics, and

does that clean R5 survive higher n? Answers: NO and YES.**

R5 x6 — THE HEADLINE: ZERO LOOPS, SIX FOR SIX

| run | complete | looped | redundant |

|---|---|---|---|

| r5 / r5c / r5d / r5e / r5f | 6/6 | 0/6 | 0.0% |

| r5b | 5/6 | 0/6 | 0.0% |

35/36 complete, 0 loops and 0.0% redundancy in every one of six runs. This is

the best long-horizon record measured on this box. Six reps was deliberate:

[[mellum2-12b-a25b-trial]] is the cautionary tale — its n=1 grill ALSO showed

R5 6/6 with 0 loops, and n=13 exposed 31% LOOPING. See

[[round5-is-a-sample-not-a-measurement]].

Agentic-candidate comparison (the reason this grill was run):

| | tommy2 (n=6) | gpt-oss-20b (n=9) | mellum (n=13) |

|---|---|---|---|

| R5 looped | 0% | loops repeatedly | 31% |

| coding r1-r4 | 22/23 | 22/23 | 21/23 |

| KLayout+ref | 7/8 | 7/8 | 0-3/8 |

vs THE Q4_K_M PARENT — what 2-bit cost, and what it did not

| suite | tommy2 (IQ2_M) | tommy (Q4_K_M) |

|---|---|---|

| coding r1-r4 | 22/23 | 21/23 (BETTER at 2-bit) |

| R5 loops | 0/6 x6 | 0/6 x2 |

| KLayout + ref | 7/8 | 7/8 (unchanged) |

| KLayout unaided | 2/8 | 1/8 |

| office + ref | 9/9 | 9/9 (unchanged) |

| office UNAIDED | 0/9 | 9/9 |

| closedloop unaided | 2/4 | 4/4 |

| realcase | 0/7 CAP-HIT runaway | 0/7 (harness crash, NOT capability) |

| envelope | CLEAN, 0 leaks / 271 turns | CLEAN |

THE FINDING: 2-BIT DESTROYED API RECALL, NOT REASONING

office 0/9 unaided is not incompetence — all three :struct stages failed on

invented or misremembered APIs, each cascading to 2 BLOCKED stages

([[office-loop-denominator-fix]]):

TypeError: ChartBase.add_data() got an unexpected keyword argument 'values'

NameError: name 'ws' is not defined

chart plots rows [(1,5)] ... Reference + titles_from_data eats one point

Hand it the reference and it scores 9/9. This is exactly the signature named

in [[gemma12-grill]]: *"its weakness is API RECALL, not capability."*

And unaided it RUNS AWAY. formula_totals:struct burned **32,000 tokens in

400 s** unaided vs 6,990 tokens WITH the reference; realcase died identically

(32,000 tokens, CAP-HIT). 6 abnormal finish_reason turns (2.2%). The runaways

appear only on the unaided arms — missing API knowledge sends it into a loop of

guessing rather than failing fast.

HOW TO USE IT

Best agentic coder on the box — but ONLY with API references in context.

For KLayout work the klayout-classes skill is NOT optional with this model; it

is the difference between 7/8 and inventing methods. Same for openpyxl/office.

Without references expect invented APIs and 32k-token runaways, not graceful

failure.

Note the parent's realcase 0/7 was a VERIFIED harness crash; tommy2's 0/7 is a

GENUINE runaway. Same score, different cause — do not copy the parent's

exoneration across.

llama-swap.yaml entry

  "tommy2":
    aliases: [thomson-1.0-small-iq3xxs-4060ti-131k]
    # SINGLE-CARD CLONE of tommy on the 4060 Ti — runs IN PARALLEL with the
    # FreeToken driver on the 5060 Ti (gemma4-ft / qwen36-ft). Same model
    # (thomsonreuters/Thomson-1.0-Small, Qwen3.6-35B-A3B) + same patched family
    # template + shared mmproj, but bartowski IQ2_M (11.24 GiB, 2-bit) instead of
    # tommy's Q4_K_M (19.92 GiB): tommy's Q4_K_M does NOT fit one 16 GiB card, and
    # IQ3_XXS (13.85) + mmproj (0.84) eats ~14.7 GiB leaving no room for KV +
    # compute on the 4060 Ti, so the 4060-Ti clone steps to 2-bit to keep vision
    # + 131k context (the only quant that fits vision+131k on one 16 GiB card).
    # Sized: 11.24 weights + 0.84 mmproj + 0.70 q4_0 KV @131k + ~1.0 compute =
    # ~13.8 / 15.26 GiB usable. Context 131072 (NOT tommy's 262k — 2-bit @ 262k
    # pushes KV+compute over the 4060 Ti ceiling). Pinned to
    # CUDA_VISIBLE_DEVICES=1 (PCI_BUS_ID = the 4060 Ti,
    # [[cuda-device-order-pci-bus-id]]), single-GPU (no --tensor-split / no TP),
    # so it coexists with the 5060 Ti FreeToken driver. Lives in the 4060 Ti swap
    # pool: evicts mellum / gemma12 / bonsai / qwen3vl-8b / glm-ocr when loaded
    # (one at a time, as the pool already does). q4_0 V-cache accepted by
    # qwen3_5_moe (qwen38 / fable-fusion prove it). REASONING MODEL — budget
    # max_tokens generously (short caps return EMPTY content mid-think, same
    # trap as tommy / pocket-35b). 2-bit static (bartowski IQ2_M, NOT unsloth
    # dynamic) — quality gamble; the box's coder-agentic uses UD-IQ2_XXS dynamic
    # (22/23), static 2-bit is the band that turned DeepSeek-R1-Distill-70B to
    # garbage ([[model-triage-checklist]] check). Grill before trusting.
    cmd: |
      /usr/bin/env
      CUDA_DEVICE_ORDER=PCI_BUS_ID
      CUDA_VISIBLE_DEVICES=1
      ${llama_bin}
      -m /mnt/ssk500/gguf/thomsonreuters_Thomson-1.0-Small-IQ2_M.gguf
      --mmproj /mnt/ssk500/gguf/mmproj-thomsonreuters_Thomson-1.0-Small-bf16.gguf
      --alias tommy2
      --jinja --chat-template-file chat-templates/qwen35moe-nonfirst-system-256k.jinja
      -ngl 99 -c 131072 -fa on
      -b 2048 -ub 512
      --temp 0.6 --top-p 0.95 --top-k 20 --min-p 0.0
      --cache-type-k q4_0 --cache-type-v q4_0
      --host 127.0.0.1 --port 9173 --parallel 1
    proxy: http://127.0.0.1:9173