Scope

• Model tested: minimax/minimax-m3
• Total tests: 459
• Categories covered: All available categories were included (coding, cost, debugging, hallucination, reasoning, refactoring, security, speed, ui)
• Run mode: Full benchmark (no limiting flags like --limit or --category used)
• Fail-fast mode: Disabled (fail_fast=false), so all tests were attempted even after failures
• Results completeness: The full results are truncated — only a subset of test outcomes is included in this payload.

Performance Summary

Overall Results

• Pass rate: 266/459 (57.95%)
• Score: 2357.02 out of 3155 (74.71%)
• Total cost: $0.3736
• Latency:
  • Average: 16.80s
  • Median: 11.75s
  • Average Time to First Token (TTFT): 1.32s
• Output speed: 46.05 tokens/sec

---

Category-Level Performance

✅ Strong Performers

• Cost:
  • Pass rate: 27/30 (90%)
  • Score: 132/150 (88%)
  • Very low cost: $0.0068
  • Fast output: 58.10 tok/s
• Coding:
  • Pass rate: 51/60 (85%)
  • Score: 241/261 (92.34%)
  • High efficiency: 74.65 tok/s
  • Cost: $0.0268
• Speed:
  • Pass rate: 49/60 (81.67%)
  • Score: 156/180 (86.67%)
  • Fast generation: 51.44 tok/s
• Hallucination:
  • Pass rate: 41/60 (68.33%)
  • Score: 291/360 (80.83%)
  • Strong resistance to hallucination, especially on medium difficulty (95% pass)

⚠️ Moderate Performers

• Security:
  • Pass rate: 34/60 (56.67%)
  • Score: 386/512 (75.39%)
  • Slower output: 30.47 tok/s
• Debugging:
  • Pass rate: 36/60 (60%)
  • Score: 420/601 (69.88%)
  • Performance degrades on hard tests (50% pass rate)

❌ Weak Performers

• Reasoning:
  • Pass rate: 17/60 (28.33%)
  • Score: 388/541 (71.72%)
  • Very low pass rate across all levels (only 15% on easy)
• Refactoring:
  • Pass rate: 10/60 (16.67%)
  • Score: 338/541 (62.48%)
  • Consistently poor across all difficulty levels (15% max pass rate)
• UI:
  • Pass rate: 1/9 (11.11%)
  • Only 1 test passed (ui::easy), all medium/hard failed
  • High cost: $0.0 of total cost attributed here despite few tests

---

Difficulty-Level Insights

• Easy tests: Generally well-handled, especially in coding (100% pass) and cost (80% pass)
• Medium tests: Performance drops in debugging, reasoning, and refactoring (45%, 45%, 15% pass respectively)
• Hard tests: Significant struggles in reasoning (25% pass), refactoring (15%), and ui (0%)

---

Notable Failures and Observations

• coding-hard-topological-sort:
  • Failed with null TTFT and latency of 120.16s — likely timed out
• reasoning category:
  • Repeated failures on constraint reasoning tasks, even at easy level (e.g., reason-constraint-disk-quota, reason-rc-login-timeout)
• refactoring:
  • Extremely low pass rate (10/60) with no level exceeding 15% success
• ui category:
  • Only ui::easy passed; all 6 medium and 2 hard tests failed
• Latency outliers:
  • Some debugging and reasoning tests exceeded 20s, with max at 34.75s (reason-constraint-consistency-latency)
• Cost concentration:
  • refactoring was most expensive category: $0.0780 (20.9% of total cost), driven by high token usage (64k completion tokens)

---

Summary

The minimax/minimax-m3 model performs strongly in coding, cost optimization, and speed tasks, showing fast response times and high accuracy. It also resists hallucination well, especially on medium-difficulty prompts.

However, it struggles significantly with reasoning, refactoring, and UI tasks, with pass rates below 17% in the latter two. The model’s high failure rate on logical and architectural reasoning, even at easy levels, suggests limitations in deep program comprehension.

Despite moderate overall score (74.7%), the high cost and latency in low-pass-rate categories indicate inefficiency when handling complex or nuanced software engineering tasks.