Scope

This benchmark run evaluated a single model, x-ai/grok-4.3, across 459 tests spanning multiple categories including coding, cost, debugging, hallucination, reasoning, refactoring, security, speed, and UI. The run was not limited ("limit":null) and did not use fail-fast mode ("fail_fast":false), meaning all tests were executed or reported. The results are truncated ("results_truncated":true), indicating not all individual test outcomes are included in this payload.

Performance

The model achieved an overall pass rate of 214/459 (0.466) and a score percentage of 67.57%. Average latency was 5.35s, with a median of 2.68s. The average time to first token (TTFT) was 1.18s, and the model generated output at an average speed of 101.04 tok/s. The total cost for the run was $0.465, consuming 82,399 prompt tokens and 165,200 completion tokens.

Category-level Patterns

• High Performance: The model excelled in cost (93.3% pass rate, 90.7% score) and speed (93.3% pass rate, 97.8% score) categories, demonstrating strong efficiency and correctness in performance-critical tasks.
• Strong Coding Ability: Coding was a relative strength with a 85% pass rate and 91.2% score, particularly on easy (100% pass) and medium (70% pass) difficulty levels.
• Poor Reasoning & Refactoring: Performance was weakest in reasoning (6.7% pass rate) and refactoring (11.7% pass rate), indicating significant difficulty with complex logical or structural code changes.
• Security & Hallucination Challenges: Security had a very low pass rate (10%) and score (48.4%), while hallucination scored 70.6% but passed only 40% of tests, showing a tendency to invent incorrect API behaviors.
• UI Strength: Despite only 9 tests, the model achieved a perfect 100% pass rate in UI, with a high score percentage of 85.6%.

Notable Failures and Observations

• Critical Errors: Two debugging tests (debug-prototype-pollution-check-v2, debug-prototype-pollution-merge-v2) failed with the error "Spread syntax requires ...iterable[Symbol.iterator] to be a function", suggesting a fundamental issue with handling certain JavaScript patterns.
• High-Cost Tests: The most expensive single test was reason-constraint-subscription-migration in reasoning, costing $0.00523 and taking 40.67s with a very slow output speed of 51.3 tok/s.
• Latency Outliers: Several reasoning tests exhibited high latency, such as reason-rc-null-pointer-call (41.34s) and reason-constraint-subscription-migration (40.67s), often correlated with high TTFT (e.g., 33.09s for the former).
• Hallucination on Edge Cases: The model failed several hallucination edge cases, such as halluc-edge-integer-overflow and halluc-edge-string-truncate, incorrectly claiming non-existent behaviors.
• Difficulty Scaling: Performance degraded significantly with difficulty in most categories. For example, in security, pass rate dropped from 15% (medium) and 10% (hard) to 5% (easy), and in refactoring, it was 20% (hard), 10% (medium), and only 5% (easy).