gpt-realtime-2.1-mini Gets You Most of the Flagship for a Third of the Price. The Catch Is Latency.

Our old Speech-to-Speech test asked whether a model could hold one tool call over a couple of turns. Every current model passes that. So it told you almost nothing — the board was a wall of 100%s and near-identical judge grades. We rebuilt it around a call that actually breaks models.

The test

One 20-turn concierge call. Seven tools on the table (weather, calendar, create-event, book-flight, currency, timezone, cancel). The model has to:

• pick the right tool from seven when the caller asks something ambiguous,
• chain — look up the city of the caller’s 3 PM meeting on the calendar, then book a flight to that city,
• recover — the booking always fails (“no flights that day”), so the model has to say so instead of inventing a confirmation,
• hold context through contradictions — the travel day changes twice, Wednesday → Thursday → Friday, and at turn 19 we ask “which city and day am I flying?” and score against the latest value,
• survive a mid-sentence interruption and pick the thread back up.

That’s the kind of call a real voice agent takes. It is not a wall of 100%s.

We stopped trusting the LLM judge

We graded the first cut with an LLM judge (gpt-4.1), the same way most voice leaderboards do. It rated all nine models between 0.90 and 1.00. It called the worst model in the field a 0.90. It gave a model that dropped two-thirds of its calls a perfect 1.00. It could not see the differences that matter.

So we scored the transcripts with eight deterministic checks — pure code, no model in the loop for the objective axes: tool selection, no false calls, chaining, error recovery, date confirmation, no invented prices, hard context-recall, and brevity. The deterministic scorer spread the same nine models across 0.55 to 1.00. Everything below comes from that scorer.

Only one model aces it

gpt-realtime is the single clean sweep — 1.00 on all eight axes, zero dropped calls, and it’s also the fastest model on the board at 494ms. If you want the best realtime voice model and price isn’t the constraint, it’s still the answer. 100% is no longer free: every other model has a specific, measurable hole.

The value pick: gpt-realtime-2.1-mini

Here’s the one to know about. gpt-realtime-2.1-mini scores 0.96 — it chains the booking, recovers from the failure, confirms dates, and nails the hard-recall ("You're flying to Seattle on Friday, July 10, 2026.") exactly like the flagship. It kept all three calls. And it runs at the mini tier — roughly a third of gpt-realtime’s price.

The catch is latency. On our separate latency run, 2.1-mini is mid-pack at 1011ms — more than double gpt-realtime’s 494ms — and it’s a touch wordy (brevity 0.76). So the honest recommendation is conditional: if end-of-turn latency is your binding constraint (fast back-and-forth phone support), pay up for gpt-realtime. If you can absorb ~1 second and you’re optimizing cost against near-flagship capability — the common case for outbound, scheduling, and assistant-style calls — 2.1-mini is the buy.

One caution: don’t confuse it with gpt-realtime-mini. That’s a different model, and while it’s the value pick on the latency board, on capability it’s weaker — it hesitates instead of chaining the booking (0.50) and dropped one call in three. The capability-value winner is specifically 2.1-mini.

Newer isn’t better — and the judge can’t tell

The deterministic scorer surfaced something the judge actively hid: the newer full-size models regressed.

• gpt-realtime-2 dropped the connection in 2 of 3 calls (at turns 3 and 17). It’s capable on the one session that finished, but its scores rest on n=1 — and the LLM judge gave it a perfect 1.00, blind to the fragility.
• gpt-realtime-2.1 lands at 0.82, stalling on confirmation in one session and refusing to commit to the day on recall.

The original gpt-realtime beats both of its successors on capability and speed.

Where the rest break (with the receipts)

• Gemini 3.1 Live is near-perfect on everything except one axis: it never commits to the updated day, looping on the “no flights” state — "we haven't found availability on Wednesday, Thursday, or Friday yet. What date would you like to try next?" Hard-recall 0.00, and the judge rated it 0.93.
• grok-voice-fast loops on confirmation and never actually fires book_flight — chaining 0.00 — and hallucinated a 2024 date. The judge gave it 0.98.
• grok-voice-think-fast is the worst overall (0.55): never calls the tools, never confirms a date, and rambles with garbled, overlapping speech (brevity 0.38).

How it was scored

Each model ran the fixed 20-turn scenario three times through the Speko gateway. The eight axes are pure-code checks on the turn records; an axis a session never reached (e.g. a model that never attempted a booking, so had no error to recover from) is marked N/A and dropped from the average, not zeroed. Latency is a separate measurement, from a fixed us-east4 vantage over 15 sessions, and is deliberately kept in its own column — it’s a real tradeoff, not part of the capability ranking. Every transcript and per-turn audio clip is archived.

The buyer’s guide

• Need the best, or the fastest? gpt-realtime. It wins capability and latency outright.
• Optimizing cost and can eat ~1s of latency? gpt-realtime-2.1-mini — 96% of the flagship’s capability, zero dropped calls, a third of the price.
• Don’t reach for the newer full-size gpt-realtime-2 / 2.1 expecting an upgrade, and don’t let an LLM-judge leaderboard pick for you — it rated this entire field 0.90–1.00.

FAQ

What is the best-value speech-to-speech model for voice agents?

gpt-realtime-2.1-mini. On our deterministic 20-turn concierge benchmark it scored 0.96 to gpt-realtime’s perfect 1.00 - chaining the booking, recovering from the failed one, nailing hard recall - with zero dropped calls, at roughly a third of the flagship’s price. The catch is latency: 1011ms versus gpt-realtime’s 494ms.

What is the difference between gpt-realtime-mini and gpt-realtime-2.1-mini?

They are different models with different strengths. gpt-realtime-2.1-mini is the capability pick: 0.96 overall, kept all three calls. gpt-realtime-mini is weaker on capability - it hesitated instead of chaining the booking (0.50) and dropped one call in three - but it is the value pick on the separate latency board.

Can I trust LLM-judge scores on voice agent leaderboards?

Be skeptical. Our LLM judge (gpt-4.1) rated all nine models between 0.90 and 1.00 and gave a perfect 1.00 to gpt-realtime-2, which dropped the connection in 2 of 3 calls. Eight deterministic code checks spread the same nine models across 0.55 to 1.00 and surfaced every hole.

Is gpt-realtime-2 an upgrade over gpt-realtime?

No. gpt-realtime-2 dropped the connection in 2 of 3 test calls (at turns 3 and 17), and gpt-realtime-2.1 landed at 0.82, refusing to commit on date recall. The original gpt-realtime swept all eight axes at 1.00 and is also the fastest model at 494ms.