Start building →Open modelsCost per solveRecommended stacksSpeech-to-TextText-to-SpeechMethodologySTTTTSLLMS2SBlogAbout

Open models

Mean per-run score over 19 scripted scenarios: 1 completed, 0 acted dangerously, part-credit if it stopped early but safely.Task doneInvented a policy or fact it was never given, % of out-of-policy questions.FabricationTurns where the caller hears silence (no spoken words), % of turns.Dead-airTime to first output on a real agent turn (system prompt + tools), p50; whisker to p90.TTFT p50
—
Cerebras
gpt-oss-120b
82.02%70–93% 95% CI bootstrap over items · n=95 · measured 2026-10-04
0%0 of 10 probes (30 probe runs) · ≤31%
70.5%227 of 322 turns
139ms139–261 p50–p90 · n=38
—
Cerebras
gemma-4-31b
—no measurement on the current corpus — transport unavailable
—fabrication probes not run for this model
—not measured on the current corpus
—not measured on the vendor-direct regional sweep
—
Together
Llama-3.3-70B
54.30%39–70% 95% CI bootstrap over items · n=95 · measured 2026-10-04
0%0 of 10 probes (30 probe runs) · ≤31%
48.5%94 of 194 turns
525ms525–725 p50–p90 · n=38
—
Together
DeepSeek-V4-Pro
—no measurement on the current corpus — transport unavailable
—fabrication probes not run for this model
—not measured on the current corpus
—not measured on the vendor-direct regional sweep

Data: CC BY 4.0, attribution "Speko Benchmarks".