Loading model…
stepfun/step-3.7-flash25/27 in v2 — Resilience, score 55.2%, pass rate 38.1% (175/459)0.68s, output speed 257 tok/s$0.74, prompt tokens 33236, completion tokens 64108568.3% pass rate, 41/60 passed)48.3% pass rate, 29/60)$0.74 total across 459 tests3.3% pass rate, only 2/60 passed)0% pass rate, 0/9)38.1%) indicates reliability gaps across suitesopenai/gpt-5.5 (77.9% score, 61.9% pass rate), openai/gpt-5.3-codex (77.7%, 61.2%), and qwen/qwen3.7-max (75.3%, 54.9%) by wide margins in both score and pass rate.Ranked 25/27 in Resilience with 55.2% score and 38.1% pass rate, stepfun/step-3.7-flash is a low-cost option suitable only for non-critical tasks where reliability is not paramount. Its strong hallucination resistance is offset by severe refactoring and UI weaknesses. Not recommended for production coding workflows requiring consistent correctness.
Overall score % from merged run_models rows (chronological). Only runs that include this model appear as points.
Score % vs pass rate % per category. With 0/1 scorers, both usually line up; with proportional tests, score % reflects partial credit while pass rate counts tests that clear the fixture threshold.
Total estimated spend per scope for this model (bars, left axis) and mean spend per merged result row (line, right axis: total ÷ tests).
Pass rate % per difficulty level — complements the score % view above.
Normalized 0–100 within this model: TTFT (shorter → higher spoke) and decode tok/s (higher → higher spoke). Values come from streamed BLXBench runs merged into overall_ranking.json.
Pass rate % per category for this model (distinct from score %, which reflects partial credit).