Open models

	Multi-step tool tasks completed end-to-end (chains, compound, clarify), % of runs.Task done	Invented a policy or fact it was never given, % of out-of-policy questions.Fabrication	Turns where the caller hears silence (no spoken words), % of turns.Dead-air	Time-to-first-token, p50 (us-east4). Whisker spans p50→p90.TTFT p50
05Rank 5 of 12 on the full LLM board — not re-ranked for this page. Cerebras gpt-oss-120b	94%	10%	51%	195ms195–260 p50–p90
01Rank 1 of 12 on the full LLM board — not re-ranked for this page. Cerebras gemma-4-31b	89% †	0%	12%	192ms192–232 p50–p90
10Rank 10 of 12 on the full LLM board — not re-ranked for this page. Together Llama-3.3-70B	56%	0%	29%	698ms698–860 p50–p90
— Together DeepSeek-V4-Pro	100%	0%	0%	—No real-time first-token measurement; exceeds the ~800ms voice latency budget.

	Average Word Error Rate over 7 English short-form test sets. Lower is better. Sourced from the Hugging Face Open ASR Leaderboard, 2026-07-16 — not measured on our rig.WER avg	Seconds of audio transcribed per second of compute (the Open ASR Leaderboard's H200 run). Higher is faster.RTFx
01 Boson AI Higgs Audio v3 STT	4.62%	110×
02 AutoArk AI ARK-ASR-3B	4.64%	484×
03 OpenMOSS (Fudan) MOSS-Transcribe-preview-2B	4.66%	149×
04 Boson AI Higgs Audio v3 8B STT v2	4.73%	139×
05 IBM Granite Speech 4.1 2B	4.90%	547×
06 IBM Granite Speech 4.1 2B NAR	4.95%	2079×
07 Alibaba (Qwen) Qwen3-ASR-1.7B	4.99%	796×
09 NVIDIA Canary-Qwen 2.5B	5.06%	861×
10 IBM Granite 4.0 1B Speech	5.10%	661×
11 AutoArk AI ARK-ASR-0.6B	5.14%	672×
12 Cohere Cohere Transcribe	5.20%	916×
13 SoundsgoodAI Zipformer XL 290M	5.21%	160×
14 IBM Granite Speech 3.3 8B	5.29%	264×
15 NVIDIA Parakeet-TDT 0.6B v2	5.39%	6038×

	Blind pairwise listening preference (which voice sounds more natural). Higher is better. Sourced from a public speech arena, 2026-07-16 — not measured on our rig.Elo
01 Fish Audio Fish Audio S2 Pro	1116
02 StepFun Step-Audio-EditX	1110
03 Mistral Voxtral TTS	1078
04 NVIDIA Magpie-Multilingual 357M	1060
05 hexgrad Kokoro 82M v1.0	1058
06 Maya Research Maya1	1048
07 Resemble AI Chatterbox	1012
09 Zyphra Zonos-v0.1	1000
10 MyShell OpenVoice v2	957
12 Coqui XTTS v2	913
13 yl4579 StyleTTS 2	889
14 MetaVoice MetaVoice-1B v0.1	835

	Correct END-vs-WAIT decisions across 200 real clips, at each model’s best operating threshold (min false-cutoff s.t. END-recall ≥ 85%).EOT accuracy	Completed turns correctly ended — the agent didn’t leave the caller hanging.END-recall	WAIT clips wrongly ended = talked over the caller mid-thought. The costly live-call error.False-cutoff	Model forward pass, warm CPU. Text models additionally wait on the STT transcript; audio runs in parallel with STT.Inference
01Rank 1 of 5 on the full Turn-taking board — not re-ranked for this page. Smart Turn v3.2 Pipecat · Whisper-tiny	94.0%	89.0%	1.0%One false cutoff in 100 incomplete clips.	32msRuns in parallel with STT — no transcript wait.49 p90
02Rank 2 of 5 on the full Turn-taking board — not re-ranked for this page. LiveKit Intl turn-detector v0.4.1	87.0%	86.0%	12.0%	29msWaits for the STT transcript (~0–125ms) before running.57 p90
03Rank 3 of 5 on the full Turn-taking board — not re-ranked for this page. Turnsense SmolLM2-135M	86.0%	88.0%	16.0%	127msPads every input to 256 tokens regardless of length — slowest as-shipped.193 p90
04Rank 4 of 5 on the full Turn-taking board — not re-ranked for this page. LiveKit EN turn-detector v1.2.2	82.0%	87.0%	23.0%	5ms24 p90