Open models

	Mean per-run score over 19 scripted scenarios: 1 completed, 0 acted dangerously, part-credit if it stopped early but safely.Task done	Invented a policy or fact it was never given, % of out-of-policy questions.Fabrication	Turns where the caller hears silence (no spoken words), % of turns.Dead-air	Time-to-first-token, p50 (us-east4). Whisker spans p50→p90.TTFT p50
— Cerebras gpt-oss-120b	85.00%70–96% 95% CI bootstrap over items · n=95 · measured 2026-08-25	30%9 of 30 probe runs	68.2%204 of 299 turns	195ms195–260 p50–p90
— Cerebras gemma-4-31b	77.72%62–92% 95% CI bootstrap over items · n=95 · measured 2026-08-25	0%0 of 10 probes (30 probe runs) · ≤31%	53.3%122 of 229 turns	192ms192–232 p50–p90
— Together Llama-3.3-70B	64.21%43–84% 95% CI bootstrap over items · n=95 · measured 2026-08-25	0%0 of 10 probes (30 probe runs) · ≤31%	74.3%260 of 350 turns	698ms698–860 p50–p90
— Together DeepSeek-V4-Pro	—no measurement on the current corpus — transport unavailable	—fabrication probes not run for this model	—not measured on the current corpus	—No real-time first-token measurement; exceeds the ~800ms voice latency budget.

	Average Word Error Rate over 7 English short-form test sets. Lower is better. Sourced from the Hugging Face Open ASR Leaderboard, 2026-07-16 — not measured on our rig.WER avg	Seconds of audio transcribed per second of compute (the Open ASR Leaderboard's H200 run). Higher is faster.RTFx
01 Boson AI Higgs Audio v3 STT	4.62%	110×
02 AutoArk AI ARK-ASR-3B	4.64%	484×
03 OpenMOSS (Fudan) MOSS-Transcribe-preview-2B	4.66%	149×
04 Boson AI Higgs Audio v3 8B STT v2	4.73%	139×
05 IBM Granite Speech 4.1 2B	4.90%	547×
06 IBM Granite Speech 4.1 2B NAR	4.95%	2079×
07 Alibaba (Qwen) Qwen3-ASR-1.7B	4.99%	796×
09 NVIDIA Canary-Qwen 2.5B	5.06%	861×
10 IBM Granite 4.0 1B Speech	5.10%	661×
11 AutoArk AI ARK-ASR-0.6B	5.14%	672×
12 Cohere Cohere Transcribe	5.20%	916×
13 SoundsgoodAI Zipformer XL 290M	5.21%	160×
14 IBM Granite Speech 3.3 8B	5.29%	264×
15 NVIDIA Parakeet-TDT 0.6B v2	5.39%	6038×

	Blind pairwise listening preference (which voice sounds more natural). Higher is better. Sourced from a public speech arena, 2026-07-16 — not measured on our rig.Elo
01 Fish Audio Fish Audio S2 Pro	1116
02 StepFun Step-Audio-EditX	1110
03 Mistral Voxtral TTS	1078
04 NVIDIA Magpie-Multilingual 357M	1060
05 hexgrad Kokoro 82M v1.0	1058
06 Maya Research Maya1	1048
07 Resemble AI Chatterbox	1012
09 Zyphra Zonos-v0.1	1000
10 MyShell OpenVoice v2	957
12 Coqui XTTS v2	913
13 yl4579 StyleTTS 2	889
14 MetaVoice MetaVoice-1B v0.1	835

	Correct END-vs-WAIT decisions across 200 real clips, at each model’s best operating threshold (min false-cutoff s.t. END-recall ≥ 85%).EOT accuracy	Completed turns correctly ended — the agent didn’t leave the caller hanging.END-recall	WAIT clips wrongly ended = talked over the caller mid-thought. The costly live-call error.False-cutoff	Model forward pass, warm CPU. Text models additionally wait on the STT transcript; audio runs in parallel with STT.Inference
01Rank 1 of 5 on the full Turn-taking board — not re-ranked for this page. Smart Turn v3.2 Pipecat · Whisper-tiny	94.0%	89.0%	1.0%One false cutoff in 100 incomplete clips.	32msRuns in parallel with STT — no transcript wait.49 p90
02Rank 2 of 5 on the full Turn-taking board — not re-ranked for this page. LiveKit Intl turn-detector v0.4.1	87.0%	86.0%	12.0%	29msWaits for the STT transcript (~0–125ms) before running.57 p90
03Rank 3 of 5 on the full Turn-taking board — not re-ranked for this page. Turnsense SmolLM2-135M	86.0%	88.0%	16.0%	127msPads every input to 256 tokens regardless of length — slowest as-shipped.193 p90
04Rank 4 of 5 on the full Turn-taking board — not re-ranked for this page. LiveKit EN turn-detector v1.2.2	82.0%	87.0%	23.0%	5ms24 p90

Data: CC BY 4.0, attribution "Speko Benchmarks".