One Attribute From a Stranger: <prosody pitch> Swaps the Speaker on Speechify

Here is a real, expressive line from Speechify’s flagship simba-3.2, voice harper_32 — “Oh my goodness, you remembered my birthday. I really wasn’t expecting this. Thank you so much!” — read plain:

Now the identical line, identical voice ID, with the SSML you’d genuinely author for it: a pause after “goodness,” emphasis on “remembered,” a slowed middle, and — on the closing — <prosody pitch="+15%" volume="+3dB">Thank you so much!</prosody>:

Listen to the last three words. The break, the emphasis and the slow-down all keep harper’s voice intact — then the pitch lifts on “Thank you so much,” and the speaker changes mid-sentence. Same voice ID, same generation. It is not the same person by the end.

To isolate what did it, we stripped the line down to the closing — “Thank you so much!” — and moved only the pitch. Plain:

<prosody pitch="+15%">:

And +40%:

Same one-second line. Not the same speaker. And notice what it is not: it is not longer, it did not “wander,” nothing accumulated. The moment the pitch transform is applied, the person changes.

This is not drift

We published a piece on Cartesia drift — a voice that decays away from itself over a two-minute take. That is a temporal failure: the longer the generation runs, the further the autoregressive state slides from its anchor. Length is the cause; the fix is a shorter take or a more stable voice.

This is a different animal. Here the audio is one second long and the identity is already gone. Nothing drifted. The <prosody pitch> transform itself replaced the speaker. Two different axes:

	Cartesia drift	The pitch trap
Trigger	elapsed time in one long take	a single SSML attribute
Audio length	~2 minutes	~1 second
Cause	autoregressive state decay	the pitch transform
Fix	shorter takes / stabler voice	don’t pitch a branded voice

Conflating them would be the mistake. Drift is about duration. This is about the transform you applied.

The number

We took one fixed utterance and generated it under six conditions, changing only the SSML wrapper. Then we embedded each with ECAPA-TDNN (the same speaker-verification model our drift probe uses) and measured cosine similarity to the plain, un-tagged voice. Duration is held constant across every condition, so length cannot be the variable — only the tag is.

SSML applied	ECAPA cosine to plain voice	Reading
(second plain take — baseline)	0.63	same voice, different generation
rate="x-slow"	0.60	same speaker
volume="+6dB"	0.69	same speaker
pitch="+15%"	0.28	halfway to a stranger
pitch="+40%"	0.02	a different speaker
pitch="-40%"	−0.04	a different speaker

Two plain takes of the same line land at 0.63 — that is our “same voice, different roll of the dice” baseline (short clips embed noisier than long ones, which is exactly why we compare within the same length rather than to an absolute threshold). rate and volume sit right on that baseline: 0.60 and 0.69. They change the delivery without touching who is talking.

pitch does not. A modest +15% drops similarity to 0.28 — less than half the baseline, already an audibly different person. At ±40% the cosine falls to ~0: in embedding space, orthogonal to the original speaker. Not degraded — replaced. And it is a clean dose-response — more pitch, more distance — so this is a mechanism, not measurement noise.

Hear the control

The controls are the proof. Here is the same line at volume="+6dB" — louder, but unmistakably the same person:

And at rate="x-slow" — slower, same person:

Put the volume and rate clips next to the pitch clips above. Volume and rate are knobs on one voice. Pitch is a voice swap.

The mechanism, briefly

A pitch shift can be done two ways. The right way for TTS is formant-preserving — move the fundamental frequency (F0) but hold the vocal-tract resonances (the formants) that encode a speaker’s identity. The cheap way is a naive scale — resample the whole signal, which drags the formants along with F0 and makes the speaker sound smaller/larger, younger/older: a different person.

Speechify’s behaviour is the signature of the cheap path. (We also tried to confirm it by tracking formants directly, but on a one-second exclamation the formant estimates were too noisy to stand on — so we lead with the ECAPA result, which is unambiguous about the outcome even where the mechanism measurement is not.) The practical point stands regardless of the internal cause: on simba-3.2, <prosody pitch> is a different-voice dial, not a same-voice-higher dial.

Caveats

• One voice, one model, one line. Measured on simba-3.2 / harper_32, English, “Thank you so much!”. A clean scope, not a survey.
• Short-clip embeddings are noisy — hence the within-length comparison to a same-length baseline and controls, not an absolute cosine threshold. The pitch conditions collapse far below both, which is the load-bearing fact.
• Measured through the Speko gateway, on the audio the gateway actually returns.
• This is a Speechify finding. Other providers may implement <prosody pitch> with proper formant preservation — we haven’t run the same probe on them yet. That comparison is the obvious next piece.

What it means for production

The rule is simple, and it is the opposite of what most people assume about SSML:

• Never use <prosody pitch> on a branded or consistent agent voice. It does not “raise the pitch a little.” It silently changes who the customer thinks they are talking to — on the very first sentence, with no drift needed.
• rate, volume, and break are safe. They control pace, level, and pauses without touching identity. If you need expressive control on a fixed voice, that is your toolkit.
• If you want a different-sounding voice, pitch is a legitimate lever — just know that’s what it is.

There is a benchmark-shaped hole here too. Leaderboards score clarity and naturalness; our drift probe scores steadiness over time. Nobody scores identity-preservation under prosody — whether a provider’s own control tags keep the speaker intact. On this test, Speechify’s pitch tag fails it, and its rate/volume tags pass. That’s a dimension worth measuring across the field.