Start building →

Request a new model

Tell us which model you want measured. We review every request.

Speech-to-Speech

Benchmarks

X
Y
600ms800ms1.00s1.20s0.600.700.80First sound lower is betterAction higher is bettergrok-voice-3.0grok-voice-think-fast-2.03.1-flash-livegpt-realtimegpt-realtime-2.1-mini3.8-live3.8-live-extended-thinkinggpt-live-1gpt-realtime-2gpt-realtime-2.1gpt-realtime-minibenchmarks.speko.ai · run 07-03

Action

higher is better

Did the agent actually fire the tool after the caller changed a constraint mid-call — rebook the flight, move the reservation.

OpenAIgpt-realtime-2Try →0.87
OpenAIgpt-realtimeTry →0.84
GeminiGemini3.8-liveTry →0.83
GeminiGemini3.8-live-extended-thinkingTry →0.83
OpenAIgpt-realtime-2.1-miniTry →0.83
GeminiGemini3.1-flash-liveTry →0.78
OpenAIgpt-live-1Try →0.78
OpenAIgpt-realtime-miniTry →0.77
xAIgrok-voice-3.0Try →0.67
xAIgrok-voice-think-fast-2.0Try →0.67
OpenAIgpt-realtime-2.1Try →0.56
0.000.250.500.751.00

Call completion

higher is better

Share of the scripted turns that came back with speech or a tool call. A missing turn is a caller left waiting.

OpenAIgpt-live-1Try →100%
xAIgrok-voice-3.0Try →100%
xAIgrok-voice-think-fast-2.0Try →100%
GeminiGemini3.8-liveTry →98%
GeminiGemini3.8-live-extended-thinkingTry →96%
OpenAIgpt-realtime-2.1-miniTry →93%
GeminiGemini3.1-flash-liveTry →92%
OpenAIgpt-realtime-2.1Try →90%
OpenAIgpt-realtimeTry →88%
OpenAIgpt-realtime-2Try →81%
OpenAIgpt-realtime-miniTry →74%
0%25%50%75%100%

Task capability

higher is better

Six checks across one call: right tool, no tool on a negative turn, error recovery, no invented prices, late recall, brevity.

xAIgrok-voice-3.0Try →0.94
OpenAIgpt-realtime-miniTry →0.93
GeminiGemini3.8-liveTry →0.92
OpenAIgpt-realtimeTry →0.92
xAIgrok-voice-think-fast-2.0Try →0.92
OpenAIgpt-live-1Try →0.90
GeminiGemini3.8-live-extended-thinkingTry →0.88
GeminiGemini3.1-flash-liveTry →0.86
OpenAIgpt-realtime-2Try →0.83
OpenAIgpt-realtime-2.1-miniTry →0.80
OpenAIgpt-realtime-2.1Try →0.76
0.000.250.500.751.00

First sound

lower is better

When the model starts talking. On a duplex model that is often filler — "let me check that" — not the answer.

OpenAIgpt-realtimeTry →494ms
OpenAIgpt-realtime-miniTry →614ms
xAIgrok-voice-3.0Try →782ms
xAIgrok-voice-think-fast-2.0Try →820ms
GeminiGemini3.8-live-extended-thinkingTry →844ms
OpenAIgpt-realtime-2.1-miniTry →1.01s
OpenAIgpt-realtime-2Try →1.10s
OpenAIgpt-realtime-2.1Try →1.10s
GeminiGemini3.1-flash-liveTry →1.11s
GeminiGemini3.8-liveTry →1.21s
OpenAIgpt-live-1Try →1.22s
0ms500ms1.00s1.50s2.00s

Answer

lower is better

When the caller hears the answer they asked for. Measured since September 2026, so earlier rows carry no value.

xAIgrok-voice-3.0Try →1.60s
GeminiGemini3.8-liveTry →2.01s
OpenAIgpt-live-1Try →4.19s
GeminiGemini3.8-live-extended-thinkingTry →7.52s
0ms2.00s4.00s6.00s8.00s

gpt-live-1 · full duplex

It listens while it speaks, so it answers the way a person on a phone does: it says something the moment it has something to say, then keeps talking once it knows more.

Shape of one GPT-Live turnyou stop speaking1.22sfirst words“Sure, I’ll check.”tool result arrives87msacknowledges“Checking.”1.75sanswers“It’s 11:30 p.m. in Tokyo.”
87ms
to acknowledge a tool result
1.75s
to speak the answer after it
1.67s
longest silence inside a turn
9 words
median reply
0.59s
to stop when talked over
18/18
calls finished

What we tested

ScenarioThe story it forcesTool
defaultbook a flight to your 3pm-meeting city, then flip the daybook_flight
travel_supportinfer the wedding city, book there, flip Thu→Sat, spell an emailbook_flight
restaurant_calendarinfer the anniversary night, make the reservation, move 7→8pmcreate_event
finance_currencyinfer trip budget + currency, convert it, then bump the budgetconvert_currency
refund_negativenegative-heavy support; infer the one free pickup day, schedule it, spell an order codecreate_event
multiconstraint_scheduleinfer the summit city, book a flight, then change destination Berlin→Munichbook_flight
Overall = mean of the six · 0–1 · higher is better · “—” = dropped every session
ModelOverallDefaultTravelRestaurantFinanceRefundMulti-con.
Geminigemini-3.8-live0.880.960.990.771.000.860.67
gpt-live-10.860.990.810.870.960.880.66
grok-voice-3.00.850.960.880.980.890.750.66
grok-voice-think-fast-2.00.840.990.870.970.890.690.62
Geminigemini-3.8-live-extended-thinking0.830.930.990.730.860.850.62
gpt-realtime0.780.960.950.910.750.460.67
Geminigemini-3.1-flash-live0.780.920.990.740.840.870.32
gpt-realtime-2.1-mini0.770.970.880.600.910.810.43

FAQ

What is the best speech-to-speech model for a voice agent?
Among native speech-to-speech models, OpenAI gpt-realtime-2 led confirmed write actions at 0.87 on a 0-1 scale. xAI grok-voice-3.0 and xAI grok-voice-think-fast-2.0 and OpenAI gpt-live-1 tied at 100% scripted-call completion, measured as the share of each call completed.
Which realtime S2S API has the lowest latency?
OpenAI gpt-realtime measured 494ms p50 from end of speech to first audio in US East, the lowest on the board; OpenAI gpt-realtime-mini followed at 614ms. The first sound can be an acknowledgement before the answer.
Do speech-to-speech models execute tool calls?
OpenAI gpt-realtime-2 scored 0.87 on a 0-1 rate of firing a write tool after caller confirmation; OpenAI gpt-realtime-2.1 scored 0.56.
How are S2S models scored on this board?
We run six scripted concierge scenarios three times per system and score actions, completion and capability separately. A separate latency probe measures end of speech to first audio, with US East p50s from 494ms (OpenAI gpt-realtime) to 1217ms (OpenAI gpt-live-1).