ariane, Gyro-M, Swift IQ3_XXS and Flash-Next IQ2_XS against Sonnet 5.5

• Gyro-M is the strongest local model here: it passes 111 of 134 webbench tasks strictly (mean 0.92), against Swift IQ3_XXS's 107 (0.91), ariane's 95 (0.81) and Flash-Next IQ2_XS's 88 (0.80). Its whole lead over Swift comes from the 34 React and Next.js tasks (29 against 24, and 16 against 11 on Next.js alone); on the first 100 tasks the two are level, 82 against 83. ariane takes 67 of the first 100 and 28 of the new 34.
• ariane ties Sonnet and Flash-Next IQ2_XS on the canonical grill at 22 of 23 in all three runs, and scores 48, 49 and 49 of 52 on vision. In the long-horizon agent round it takes 5 of 6 in each run with zero loops; its one miss is test_driven_loop, which it fixes without ever re-running the test suite.
• Gyro-M is the only local model to complete the long-horizon agent round 6 of 6 in all three runs, as Sonnet does; on vision it scores 52, 49 and 50, and its best run is the only perfect vision score on this page. Swift IQ3_XXS and Flash-Next IQ2_XS score 49 to 50 on vision; on the grill Swift scores 19 to 22.
• The cost is speed. Gyro-M writes as much as the other Strata models (a median of about 25,700 output tokens per task, most of it thinking) but decodes at 68 tok/s median against ariane's 66 and Swift's 95; a task takes a median 380 s against ariane's 208 s and Swift's 262 s, and the whole webbench 13.1 hours (including a 14-minute pause) against ariane's 9.2.
• ariane's 39 webbench misses skew toward over-thinking: it ran with no thinking budget, so 10 of its 12 no-build answers stopped at the 32k output cap mid-answer, and 16 more built a working page that scored below 0.80.
• Each local model ran one webbench sample, Gyro-M at temperature 1.0 and Swift always picking the most likely token (see the note under the scoreboard), so a gap of a few tasks between them may not hold up on a second sample.
• Sonnet 5.5 is still well ahead at 129 of 134, with a 0.99 mean, and it finishes each task in about 18 s. Its times and token counts come through the API proxy, so they aren't comparable with the local runs.

Scoreboard

The local models ran one webbench sample each, with no temperature sent, so each uses its server's own default. Swift's webbench and grill ran at its 131k context; it was raised to 200k afterwards. The three Strata models ran with a 24k-token reasoning budget; ariane ran with none, so a task it over-thinks runs to the 32k output cap. Sonnet's column is its first of three samples. Sonnet's token counts come through the API proxy and aren't comparable with the local models. Gyro-M's webbench was paused for 14 minutes during task A-029, so that task's wall time (1,022 s) and the suite's wall hours include the pause; the engine measured 388 s for it.

Sampling differs between the Strata models. With no temperature sent, Strata engine 0.1.40 always picks the most likely token, while engine 0.1.47 (Strata 0.2.2) applies the model card's sampling: temperature 1.0, top_p 0.95, top_k 20. Swift ran everything on 0.1.40. Flash-Next IQ2_XS answered webbench tasks A-001 to A-065 (62 tasks) on 0.1.40, and the other 72 tasks, the grill and the vision test on 0.1.47. All of its React and Next.js tasks were therefore sampled at temperature 1.0. Same weights and settings on both engines. Gyro-M ran everything on 0.1.47, so every one of its answers was sampled at temperature 1.0.

Webbench by category

Strict passes out of the tasks in each category. Bars are drawn to the same 0–100% scale.

Where the local models differ

Counted over the 134 tasks all four local models have finished.

How each model fails on webbench

Canonical grill, task by task

Each square is one run, three runs per model. Highlighted rows are the tasks where the models differ.

expr_eval rejects inputs like 1++2; the local models miss that edge case almost every run, as nearly every model tested here does. Sonnet's one miss is damerau_levenshtein, which every local model passes.

All 134 webbench tasks

Score is the functional score of the first sample (1.00 = every check passes). Green is a strict pass, amber a near miss at 0.80 or above, red means nothing built. Sonnet runs: ● pass, ○ fail, for each of its three samples.

Sonnet 5.5, ariane and Swift on SGLang

The same Swift-1.5 Qwen3.8-27B NVFP4 weights served two ways: ariane is vLLM 0.31, "Swift on SGLang" is SGLang 0.5.22 with its own MTP draft head. Both split the model across two RTX 5060 Ti cards. Sonnet 5.5 is the reference. Measured 2026-10-27 with the same webbench, grill and lm-eval harness.

Same weights, same score; different tasks. Over the 134 webbench tasks both have finished: 74 passed by both, 21 by ariane only, 22 by Swift on SGLang only, 17 by neither. With one sample each, the two lists differ by sampling noise, so the one-task gap in total passes means nothing. SGLang built more of its projects (129 against 122), hit the output cap half as often and decoded faster per task, but the pass count did not follow.

Sampling differs: webbench sent no temperature to either server; Swift on SGLang then runs the model card's temperature 1.0 (top_p 0.95, top_k 20) with reasoning effort set to the maximum. The grill ran at temperature 0.2 for ariane and at the server default for Swift on SGLang, so the grill rows are not exactly like for like. Neither server had a reasoning budget. SGLang needed a patched launch (--enable-linear-replayssm-spec, single request, 0.98 memory fraction) to fit MTP at all; without MTP its pool grows to 239,516 tokens at about 38 tok/s.