gpt-realtime Is the Fastest S2S Model. It Took a Full Conversation to See It.
We retired the old two-turn latency probe and rebuilt the Speech-to-Speech board as a single live ten-turn conversation — real audio, a tool call, a mid-utterance barge-in, and a late memory callback, graded on latency and LLM-judged behavior over n=15 sessions from us-east4. The ranking moved. gpt-realtime now leads on speed at 494ms p50; gpt-realtime-mini matches its quality for a third of the price; and gpt-realtime-2.1-mini — the model this post first crowned — is now mid-pack at ~1011ms. Latency was the old story. Instruction-following is the new one.
When this post first went up, it crowned gpt-realtime-2.1-mini the fastest speech-to-speech model on our board — 448ms, the cheapest OpenAI tier, tool call intact. That number came from a two-turn probe: real audio in, one conversational turn, one required tool call, six sessions each. It was a fair way to rank first-audio latency, and it was too short to be the whole truth about a phone call. So we rebuilt the test. The fastest model changed.
What we changed
The old probe answered one question — “how fast does the first audio come back on a cold turn?” A real call asks a harder one: does the model stay fast, stay on-persona, keep using its tools, and remember what you told it, ten turns deep?
So each model now runs one live 10-turn conversation through the Speko gateway: normal turns, a tool call with the result fed back, a mid-utterance barge-in, and a late callback that probes memory. Conversational latency is still time from end-of-speech to first audio back — but now it’s the p50 over the clean, non-interrupted turns, pooled across 15 sessions (the board carries p90 and p95 alongside it). Same fixed vantage as every other row, us-east4. On top of latency, an LLM judge grades what actually happened: task success, instruction-following, tool use, and context retention.
That one change — from a cold two-turn ping to a full conversation where the audio context accumulates turn over turn — is why the numbers moved. The 2-turn probe caught each model at its best-case opening. Pooling clean turns across a full call surfaces the steady-state latency a caller actually feels deeper in, and the smaller models don’t hold their early lead there. This isn’t a walk-back; it’s the benchmark getting closer to a real call.
The new number
gpt-realtime posts 494ms — the fastest conversational latency on the board, and it’s not a lucky median: p90 lands at 768ms, p95 at 870ms. It pairs that with 100% task success and clean quality grades across the board (0.81 instruction-following, a perfect 1.00 on both tool use and context retention).
Right behind it, gpt-realtime-mini is the value pick — 614ms p50 (p90 812 · p95 847), essentially identical quality (0.82 instruction-following, 1.00 tool use), at $10 / $20 per 1M audio tokens — roughly a third of the full tier’s $32 / $64. If you’re cost-sensitive, this is the row to reach for; you give up about 120ms of median latency and nothing that shows up in the grades.
| Rank | Model | Conv. latency (p50) | Task | Instruction | Audio price |
|---|---|---|---|---|---|
| 1 | gpt-realtime | 494ms | 100% | 0.81 | $32 / $64 |
| 2 | gpt-realtime-mini | 614ms | 99% | 0.82 | $10 / $20 |
| 3 | Inworld Realtime | 973ms | 90% | 0.45 | ~$0.01 /min |
| 4 | grok-voice-think-fast | 1007ms | 100% | 0.50 | $0.05 /min |
| 5 | gpt-realtime-2.1-mini | 1011ms | 99% | 0.75 | $10 / $20 |
| 6 | gpt-realtime-2 | 1103ms | 100% | 0.85 | $32 / $64 |
| 7 | gpt-realtime-2.1 | 1104ms | 100% | 0.81 | $32 / $64 |
| 8 | Gemini 3.1 Live | 1108ms | 100% | 0.80 | $0.005 / $0.018 /min |
| 9 | grok-voice-fast | 1111ms | 100% | 0.81 | $0.05 /min |
Latency from us-east4, multi-turn, n=15, measured 2026-07-08; p50 shown, board carries p90 · p95. Prices in each vendor’s own denomination — OpenAI/Alibaba per 1M audio tokens (in/out), Gemini/xAI per minute, Inworld the TTS component only. Qwen3-Omni is held out of the ranking (see caveats). Full board at /#s2s.
Where 2.1-mini landed — and a surprise about “newer”
The model this post first crowned, gpt-realtime-2.1-mini, is now mid-pack — 1011ms p50, fifth on the board. Fast on a cold two-turn probe, ordinary across a full conversation. It’s still a perfectly good, cheap model at 99% task success; it just isn’t the speed leader anymore.
The more interesting finding is what happened to the newer full tiers. gpt-realtime-2 (1103ms) and gpt-realtime-2.1 (1104ms) run about 2× slower than the original gpt-realtime (494ms) — and the quality barely moves to pay for it. gpt-realtime-2 edges instruction-following by four hundredths (0.85 vs 0.81); gpt-realtime-2.1 matches the original exactly at 0.81. Paying double the median latency for a rounding-error quality difference is a genuine, non-obvious result — and the kind of thing a two-turn probe would never have shown you. If your agent leans on the reasoning tier for multi-step logic, the -2 / -2.1 rows are there; for the bread-and-butter voice task, the original is both faster and just as sharp.
Latency was the old story. Instruction-following is the new one.
With a full call to grade, task success is near-universal — 90% to 100% across every model that completes the job. It no longer separates anyone. The axis that does is instruction-following: adherence to the system prompt — persona, brevity, and using the tool instead of answering from memory. It ranges from 0.45 (Inworld) to 0.85 (gpt-realtime-2), and that spread is where these models actually differ.
It also reframes the “fast” question. Inworld (973ms) and grok-voice-think-fast (1007ms) clock in ahead of a couple of the OpenAI tiers on raw latency — but they score 0.45 and 0.50 on instruction-following. A model that answers quickly while drifting off-persona or answering from memory instead of calling your tool isn’t the one you want on a real call. Speed only counts once the model does what you told it to.
Where we keep ourselves honest
A benchmark that only delivers good news is marketing. Four caveats.
n = 15. Better than the old six, still modest. It’s enough to rank a clear leader across a half-second spread and to trust the order of the board; it is not a tight p99. Read the gaps between models, not the exact millisecond.
One vantage. Latency is measured from us-east4 so the models rank fairly against each other. Your callers’ distance to the datacenter rides on top of every number here — trust the gap, not the absolute.
Qwen3-Omni is held out. It verbalizes its tool calls — it speaks get_weather(...) instead of emitting a structured function call — so no tool ever fires. We confirmed that directly against DashScope across every tools-format and tool_choice combination; it’s a real capability gap, not a harness bug. Its clean multi-turn latency re-measure is still in progress, so it’s off the ranked table for now.
Gemini’s barge-in is unscored. Gemini 3.1 Live emits no interruption signal over the gateway and sends no assistant transcript frames, so we judged it via an STT-fallback transcript and left its barge-in turn unscored. That’s a gateway normalization gap on our side, not a model failure — its other grades are strong.
The takeaway
For a voice agent that has to hear a caller, answer without dead air, use its tools, and remember the conversation, gpt-realtime is the one to reach for first — the fastest model on the board with clean grades everywhere. If price is the constraint, gpt-realtime-mini gives you the same quality for a third of the cost. The mini crown didn’t disappear; it moved down the family — and it took a full conversation to see that. See the live board at #s2s.