Why it matters

Modern TTS quality enables new products (audiobooks, assistants, dubbing). Understanding options guides selection.

Advertisement

The architecture

Two-stage: text → mel-spectrogram → waveform. Or end-to-end.

Vocoder: converts mel-spectrogram to raw audio. HiFi-GAN, WaveNet.

Neural TTS pipelineText encoderphonemes/graphemesAcoustic modelmel-spectrogramVocoderwaveformEnd-to-end models (VITS) skip explicit vocoder; simpler but harder to train
TTS stages.
Advertisement

How it works end to end

Voice cloning: XTTS, VALL-E clone voice from 3-10 seconds of reference audio.

Commercial: ElevenLabs, PlayHT top quality. Coqui + open-source options too.

Latency: important for interactive. Streaming TTS emits partial audio as text arrives.

Prosody: naturalness of pitch, timing. Best models near-human.