What is the best LLM for a real-time voice agent?

Gemini gemini-3.5-flash-lite led with a 94.74% graded task score, including partial credit, while giving no spoken lead-in on most tool-call turns. Anthropic Claude Haiku 4.5 scored 81.67% and omitted a spoken lead-in on 3.3% of tool-call turns.

Which LLM has the lowest first-token latency?

Cerebras gpt-oss-120b had the lowest US East vendor-direct first-output median on a full agent turn (system prompt and tools) at 139ms. In the separate task benchmark, it gave no spoken lead-in on 100.0% of tool-call turns.

How fast does an LLM need to respond for voice?

On a full agent turn, US East vendor-direct first-output p50s range from 139ms (Cerebras gpt-oss-120b) to 2.41s (OpenAI gpt-5-nano). Turn detection and speech synthesis add to the caller's wait.

Which LLMs fabricate answers in a voice agent?

Fabrication is inventing a policy or fact the agent was never given. OpenAI gpt-4.1-mini measured 3% on out-of-policy questions and Alibaba qwen-turbo 90%, while Anthropic Claude Haiku 4.5 measured 3%.

What is dead air in a voice agent?

Dead air is a turn where the caller receives no speech. Anthropic Claude Haiku 4.5 produced dead air on 2.0% of turns, compared with 70.5% for Cerebras gpt-oss-120b.

LLM · Speko Benchmarks

Benchmarks

X

Y

Pareto frontier

Task done

higher is better

Did the agent finish the job without doing damage — look the order up, refund the right amount, and leave alone what it must not touch?

Try →94.74%

Try →93.16%

Try →92.63%

Try →91.75%

Try →90.88%

Try →89.12%

Try →86.84%

Try →86.14%

Try →85.79%

85.26%

Try →84.21%

Try →83.51%

Try →83.33%

82.46%

Try →82.19%

Try →82.02%

Try →81.93%

Try →81.67%

Try →80.88%

Try →80.53%

Try →77.02%

Try →75.61%

Try →54.30%

Try →30.61%

0%25%50%75%100%

Consistency

higher is better

Same task, five tries. The share of scenarios the agent got right every single time.

gemini-3.5-flash-lite95%

DeepSeek-V4-Flash-073189%

gpt-4.1-mini89%

gemma-4-31b84%

gpt-4.184%

Grok-4.384%

DeepSeek-V4.1-Flash68%

Qwen3.8-27B68%

gpt-5.4-mini68%

gpt-oss-120b68%

Claude Haiku 4.563%

gpt-5-mini63%

GLM-4.758%

gpt-oss-120b58%

DeepSeek-V4.1-Flash58%

gpt-5.258%

gpt-558%

gpt-5-nano53%

qwen-turbo26%

every run · some runs · never

Fabrication

lower is better

The agent states a fact it was never given — an opening time, a delivery window, a policy. It sounds confident and it is invented.

20 models0 of 30≤31% · 10 independent0%

qwen-turbo Try →27 of 3090%

Nemotron-3-Ultra Try →3 of 3010%

gemini-3.5-flash-lite Try →3 of 3010%

gpt-4.1 Try →3 of 3010%

gpt-5.2 Try →3 of 3010%

gpt-5 Try →2 of 307%

Claude Haiku 4.5 Try →1 of 303%

gpt-4.1-mini Try →1 of 303%

Spoken leaks

lower is better

Share of calls where the caller heard text meant for the system: a tool call, JSON, or the model's own reasoning, read aloud.

20 models0 of 1800%

gpt-oss-120b Try →57 of 18032%

gpt-oss-120b Try →15 of 1808%

Llama-3.3-70B Try →9 of 1805%

gpt-5 Try →8 of 1804%

gpt-5-mini Try →3 of 1802%

Qwen3.8-27B Try →2 of 1801%

gpt-5-nano Try →2 of 1801%

qwen-turbo Try →1 of 1801%

Silent turns

lower is better

How often the caller hears nothing at all — every turn counted, not just the ones where a tool runs. The small number is the narrower measure: the share of tool calls the agent made in complete silence, which for 11 of the 26 measured models is all of them.

Claude Haiku 4.5 Try →2%3% of tool calls5 of 252 turns

GLM-4.7 Try →6%10% of tool calls16 of 268 turns

qwen-turbo Try →33%100% of tool calls ≥69%49 of 149 turns

DeepSeek-V4.1-Flash Try →34%50% of tool calls109 of 318 turns

Claude Sonnet 5 Try →35%53% of tool calls101 of 292 turns

DeepSeek-V4.1-Flash Try →37%53% of tool calls117 of 320 turns

DeepSeek-V4-Flash-0731 Try →37%53% of tool calls116 of 315 turns

Qwen3.8-27B Try →43%62% of tool calls139 of 320 turns

Llama-3.3-70B Try →49%100% of tool calls ≥82%94 of 194 turns

Grok-4.3 Try →57%87% of tool calls165 of 290 turns

Claude Haiku 5.559%89% of tool calls177 of 299 turns

gpt-4.1-mini Try →61%97% of tool calls167 of 272 turns

gpt-4.1 Try →62%98% of tool calls166 of 270 turns

gpt-5.4-mini Try →62%99% of tool calls166 of 267 turns

gemini-3.5-flash-lite Try →63%99% of tool calls173 of 274 turns

gemma-4-31b64%91% of tool calls174 of 273 turns

gpt-5.6-terra Try →64%99% of tool calls181 of 283 turns

Nemotron-3-Ultra Try →64%96% of tool calls195 of 304 turns

gemini-3.7-flash Try →65%100% of tool calls ≥82%182 of 282 turns

gemini-3.8-flash Try →66%100% of tool calls ≥82%190 of 290 turns

gpt-5.6-luna Try →68%100% of tool calls216 of 317 turns

gpt-5 Try →70%100% of tool calls ≥82%223 of 321 turns

gpt-oss-120b Try →70%100% of tool calls ≥82%224 of 319 turns

gpt-oss-120b Try →71%100% of tool calls ≥82%227 of 322 turns

gpt-5.2 Try →71%100% of tool calls ≥82%245 of 345 turns

gpt-5-mini Try →72%100% of tool calls ≥82%238 of 331 turns

Mercury Voice Try →73%100% of tool calls ≥82%244 of 335 turns

gpt-5-nano Try →73%100% of tool calls ≥82%235 of 321 turns

0%25%50%75%100%

Time to first token

lower is better

How long a real agent turn — full system prompt and tools — takes to start answering.

Cerebrasgpt-oss-120b Try →139ms · 261ms

BasetenGLM-4.7 Try →148ms · 161ms

Hoppergemma-4-31b150ms · 158ms

TogetherDeepSeek-V4.1-Flash Try →157ms · 793ms

BasetenNemotron-3-Ultra Try →245ms · 299ms

CerebrasQwen3.8-27B Try →246ms · 592ms

OpenAIgpt-5.4-mini Try →390ms · 477ms

AnthropicClaude Haiku 4.5 Try →391ms · 574ms

Alibabaqwen-turbo Try →430ms · 544ms

InceptionMercury Voice Try →443ms · 1.29s

BasetenDeepSeek-V4.1-Flash Try →458ms · 1.06s

OpenAIgpt-4.1 Try →462ms · 605ms

OpenAIgpt-4.1-mini Try →480ms · 755ms

Gemini3.5-flash-lite Try →500ms · 621ms

Basetengpt-oss-120b Try →504ms · 1.22s

TogetherLlama-3.3-70B Try →525ms · 725ms

OpenAIgpt-5.6-luna Try →546ms · 683ms

BasetenDeepSeek-V4-Flash-0731 Try →558ms · 1.03s

AnthropicClaude Haiku 5.5579ms · 1.16s

OpenAIgpt-5.6-terra Try →580ms · 731ms

OpenAIgpt-5.2 Try →664ms · 825ms

Gemini3.8-flash Try →1.21s · 2.90s

Gemini3.7-flash Try →1.38s · 2.65s

AnthropicClaude Sonnet 5 Try →1.47s · 2.74s

OpenAIgpt-5-mini Try →1.73s · 5.94s

OpenAIgpt-5 Try →1.86s · 8.35s

xAIGrok-4.3 Try →2.38s · 5.38s

OpenAIgpt-5-nano Try →2.41s · 6.87s

0ms500ms1.00s1.50s2.00s2.50s

p50 p90

Cost

lower is better

What the model charges for a million tokens of its own output, at each vendor's published list price.

$0.20

$0.26

$0.40

$0.50

$0.75

$1.04

$1.20

$1.49

$1.60

$2.00

$2.20

$2.40

$2.50

$3.75

$3.96

$4.05

$4.50

$5.00

$8.00

$10.00

$12.00

$14.00

Real-call success

higher is better

Share of full calls handled cleanly: the caller got what they called for, was told the true outcome, and never heard a tool call read aloud. Callers change their minds and tools sometimes fail.