Scope

• Model tested: moonshotai/kimi-k2.6
• Total tests: 373
• Categories: 7 (coding_ui, debugging, hallucination, reasoning, refactoring, security, speed)
• Run mode: Full benchmark (no --limit or --fail-fast enforced; fail_fast was false)
• Results completeness: results_truncated is true, indicating not all per-test details are included in the payload.

Performance

• Overall pass rate: 57/373 (15.28%)
• Total score: 57.85 out of 375 (15.43%)
• Average latency: 10.97s
• Median latency: 6.62s
• Average TTFT (Time to First Token): 19.90s
• Average output speed: 1199.51 tok/s
• Total cost: $0.309

Category-level Patterns

Stronger Categories

• Security: Highest pass rate at 14/60 (23.33%) with score_percent of 23.33.
• Hallucination: Second-best at 11/60 (18.33%) pass rate, with high output speed (5010.4 tok/s).
• Debugging & Refactoring: Both at 9/60 (15%) pass rate, but debugging had better hard-level performance (6/20 vs 5/20).

Weaker Categories

• Reasoning: Lowest pass rate at 3/62 (4.84%) and lowest score_percent (6.25%). Only 1 of 20 hard tests passed.
• Coding UI: 1/6 passed (16.67%), but 2 tests timed out (including breakout_game and neon_sign_flicker at easy level).
• Speed: 10/65 (15.38%) pass rate, with speed::medium performing worst (1/21, 4.76%).

Notable Failures and Observations

• Timeouts:
  • coding_ui::breakout_game (hard) and neon_sign_flicker (easy) failed with "The operation timed out.", indicating potential issues with handling visual or complex UI generation tasks.
• High TTFT:
  • Average TTFT (19.90s) is unusually high compared to latency (10.97s), suggesting model struggles with initial response generation.
  • coding_ui::easy had the highest average TTFT: 166.91s.
• Cost Efficiency:
  • Despite high output speed in hallucination (5010.4 tok/s), cost per test remained low due to small token counts.
  • reasoning::easy had a very low cost per test (e.g., reasoning_easy_09_even_check cost only $0.00001881).
• Output Speed Variance:
  • Ranged from 74.02 tok/s in security to 5010.4 tok/s in hallucination, indicating category-specific performance divergence.

Summary

The moonshotai/kimi-k2.6 model shows modest performance overall (15.4% score), with notable strengths in security and hallucination detection, but significant weaknesses in reasoning and coding UI tasks. High TTFT and timeouts suggest latency bottlenecks, especially on complex or visual prompts. The model is cost-effective per test but struggles with consistency across difficulty levels.