Scope

• Model tested: openai/gpt-5.5
• Total tests run: 459
• Categories covered: 9 (coding, cost, debugging, hallucination, reasoning, refactoring, security, speed, ui)
• Run mode: Full benchmark (no --limit or --fail-fast used)
• Results completeness: results_truncated: true — full per-test details are not present in this payload.

Performance Summary

• Overall pass rate: 284/459 (61.87%)
• Overall score: 2443.87 / 3137 (77.90%)
• Total cost: $6.78
• Average latency: 6.06s
• Median latency: 3.95s
• Average time to first token (TTFT): 1.09s
• Average output speed: 102.68 tok/s

Category-Level Performance

High-Performing Categories

• Coding: 58/60 passed (96.67%), score 257/261 (98.47%) — strongest category.
  • Perfect on hard and easy sublevels (100% score).
• Speed: 52/60 passed (86.67%), score 161/180 (89.44%)
• Cost: 29/30 passed (96.67%), score 137/150 (91.33%)
• UI: 9/9 passed (100%), score 7.87/9 (87.49%)

Medium-Performing Categories

• Security: 32/60 passed (53.33%), score 383/512 (74.80%)
• Debugging: 38/60 passed (63.33%), score 449/583 (77.02%)
  • Performance improves with difficulty: hard (75% pass) > medium (65%) > easy (50%).
• Hallucination: 34/60 passed (56.67%), score 284/360 (78.89%)
  • Mixed results across subtypes (API, behavior, edge cases).

Lower-Performing Categories

• Reasoning: 20/60 passed (33.33%), score 412/541 (76.16%)
  • Very low pass rate despite moderate scoring — suggests partial credit on many failures.
• Refactoring: 12/60 passed (20%), score 353/541 (65.25%)
  • Poor performance across all difficulty levels, especially easy (10% pass).

Notable Observations

• High cost contributors:
  • refactoring was most expensive category: $1.62
  • ui surprisingly high cost: $1.48, driven by large completion tokens (49.5k).
• Latency outliers:
  • Highest average TTFT in reasoning (2.48s), likely due to complex constraint analysis.
  • refactoring, security, and debugging have lowest TTFT (~0.78s), suggesting fast starts but not always correct completions.
• Output speed variation:
  • Fastest: reasoning (148.47 tok/s)
  • Slowest: security (67.55 tok/s)
• Failures with errors:
  • Two debugging tests (prototype-pollution-check-v2, prototype-pollution-merge-v2) failed with runtime error: "Spread syntax requires ...iterable[Symbol.iterator] to be a function" — likely model hallucinated invalid syntax.
• Scoring anomaly:
  • In debugging::nullish-cache-hit-v2, model scored 7/10 but passed false — indicates partial credit scoring in use.

Conclusion

openai/gpt-5.5 excels in coding, cost optimization, and speed tasks, with near-perfect performance on algorithmic and efficiency challenges. It struggles significantly with reasoning under constraints and code refactoring, and shows inconsistent behavior on hallucination avoidance. The high cost in ui and refactoring suggests verbose or inefficient outputs in those domains.