Speech-to-Speech Got Smart. It Still Can't Replace the Cascade.
Speech-to-speech models closed the reasoning gap with text LLMs in 2026. The gap that's left — observability, cost predictability, component swap — is the one that actually decides production architecture.
One model or three? Every voice-agent team hits this fork. The cascaded stack runs three models in sequence — speech-to-text, an LLM, then text-to-speech. A speech-to-speech (S2S) model collapses all three into one: audio in, audio out. It’s faster, it sounds more human, and — the part that changed in 2026 — it’s no longer dumber.
We went looking for the reason not to switch, and found one. It just isn’t the one most people would guess.
The reasoning gap closed
For most of 2024 and 2025, the case against S2S was that it was dumber: sending audio straight to audio skipped the text reasoning layer, and it showed on anything multi-step.
That’s out of date. Top S2S models now score 93–98% on Big Bench Audio, the standard audio-reasoning benchmark — level with frontier text LLMs. GPT-Realtime-2 ships GPT-5-class reasoning with adjustable effort; Grok Voice and Gemini’s native-audio models sit in the same band. If you’re still telling people “S2S can’t reason,” you’re working from old information.
So if reasoning is solved, what keeps cascaded alive?
The gap that didn’t close
Three things, none about intelligence.
Observability. A cascaded stack exposes exact text at every hop — what the agent heard, what the LLM decided, what it said — and you can inspect or block a response before the customer hears it. S2S is a black box: audio never becomes text inside the pipeline, so the only transcript you get is one you generate afterward, by running STT on a recording after the call. When an agent says something wrong, the first question everyone asks is “what did it actually say?” Cascaded answers instantly; S2S can’t. In healthcare, finance, and legal, that ends the discussion.
Cost predictability. Cascaded cost is flat per turn and scales linearly with call length. S2S cost doesn’t: the full audio context re-accumulates every turn, so it grows super-linearly as the call runs long. A two-minute demo looks cheap; a twelve-minute call doesn’t — by the fortieth exchange, every turn feeds the previous thirty-nine back into the model and pays for it again.
Prompt caching flattens most of it (the dashed line), but the uncached curve is the trap: budget off a two-minute demo and a real call will surprise you. For a high-volume support line, predictable beats cheap-on-a-good-day.
Component swap. Cascaded lets you pick the best STT, LLM, and TTS independently, and swap any one when something better ships. S2S is all-or-nothing — one vendor for the whole pipeline, and the field of S2S vendors is short.
Where S2S genuinely wins
Latency. S2S removes two network hops and two model invocations: roughly 250–500 ms end-to-end versus 750–900 ms for a well-built streamed cascade. (Vendors quote a 7× advantage by comparing against a non-streamed cascade — a strawman. A competent streamed cascade is sub-second; the real gap is closer to 2×.)
Naturalness. Because audio never round-trips through text, S2S keeps prosody, emphasis, timing, and emotion. A cascade flattens all of it to text at the STT boundary. For coaching, companionship, tutoring, and premium concierge, that’s a real moat.
Barge-in. S2S handles interruptions natively through server-side duplex turn detection. A cascade can match it only by bolting on an engineered VAD and turn-detection layer — the machinery we pulled apart in our barge-in test.
When each wins
The verdict
No — S2S can’t fully replace the cascaded stack today, not for production voice agents broadly.
The 2024 reasoning objection is dead. The operational ones aren’t: observability, cost predictability on long calls, component swap, and compliance-grade auditability still favor cascaded. What’s winning in production right now is hybrid — S2S for the fast, simple turns, a cascaded path for the complex, regulated, or tool-heavy ones.
Full replacement is a 12-to-24-month question, and what gates it is no longer model quality. It’s tooling — evaluation, debugging, and compliance infrastructure for a pipeline you can’t currently see inside.
We benchmark both at speko.ai — latency, cost, and reliability by use case. If you’re choosing between one model and three, start there.