LLM
| # | Model | Score | Task done | Fabrication | Dead-air | TTFT | Cost / 1M |
|---|---|---|---|---|---|---|---|
| 1 | gemma-4-31bCerebras⚠ | 92 | 89% † | 0% | 12% | 192ms | — |
| 2 | Claude Haiku 4.5Anthropic⚠ | 91 | 100% | 0% | 0% | 532ms | $5.00 |
| 3 | gpt-4.1OpenAI⚠ | 88 | 94% | 0% | 9% | 484ms | $8.00 |
| 4 | gpt-4.1-miniOpenAI⚠ | 83 | 100% | 20% | 7% | 592ms | $1.60 |
| 5 | gpt-oss-120bCerebras⚠ | 80 | 94% | 10% | 51% | 195ms | $0.75 |
| 6 | gpt-5OpenAI | 78 | 89% | 4% | 18% | 635ms | $10.00 |
| 7 | gpt-5-nanoOpenAI | 74 | 56% | 3% | 0% | 525ms | $0.40 |
| 8 | Claude Sonnet 5Anthropic⚠ | 68 | 89% | 0% | 7% | 1.21s | $15.00 |
| 9 | gpt-5-miniOpenAI | 64 | 61% | 0% | 25% | 746ms | $2.00 |
| 10 | Llama-3.3-70BTogether⚠ | 62 | 56% | 0% | 29% | 698ms | $1.04 |
| 11 | qwen-turboAlibaba⚠ | 49 | 33% | 40% | 29% | 483ms | $0.20 |
| 12 | Grok-4.3xAI⚠ | 43 | 100% | 0% | 24% | 2.15s | $2.50 |
| — | DeepSeek-V4-ProTogether⚠ | — | 100% | 0% | 0% | — | $3.48 |