Project

Voice AI pipelines

Narrated video from a text script, built on open-weight speech models running locally on a CPU-only machine. No hosted voice APIs.

I picked the engines and voices and judged the output by ear. Claude Code agents wrote the code, inside the framework I run them in.

  • Kokoro TTS
  • faster-whisper
  • Chatterbox Turbo
  • MusicGen
  • ffmpeg
  • CPU only
KhaleejKeys Reels

A script goes in and a narrated 1080x1920 Instagram Reel comes out, for KhaleejKeys. Kokoro reads it in the af_heart voice, one take per sentence. I tried phrase-by-phrase takes first and the joins broke the prosody. Pacing is now set afterwards with a time-stretch.

Before you pay a deposit
Why the average rose

faster-whisper transcribes the voice track with word timestamps, and those timestamps drive the captions and the on-screen reveals. The transcript is also the check. Every render is matched against the script word by word: recent ones came back 91 of 91, 78 of 78 and 74 of 74.

Brand and place names need help, and the pronunciations are my call. "Khaleej" is forced to kuh-LEEJ and "Liwan" to LEE-wan with phoneme-level overrides, and the render fails if an override doesn't apply. A fix that silently stops working sounds fine until someone local hears it.

One bug is worth recording. faster-whisper's array input assumes 16 kHz audio and doesn't resample, so Kokoro's 24 kHz output came back with every word time stretched by 1.5x. The voice track is now resampled to 16 kHz before transcription.

Under the voice sits a MusicGen bed, ducked by about 11 dB while the voice speaks. The final mix is normalised to -14 LUFS. Every caption says "Narration: AI-generated voice."

The rest are on KhaleejKeys on Instagram.

Choosing a voice

I compared several open-weight engines on the same CPU box, Qwen3-TTS and Orpheus among them, and picked from samples by ear. Licences came first: anything with non-commercial weights was ruled out before it got near published work. Kokoro stayed.

A clone of my own voice

I also cloned my own voice locally with Chatterbox Turbo, from a reference recording I read myself. The reference is loudness-normalised and the output is sped up with a pitch-preserving time-stretch. It took about 26 seconds of CPU time to produce 5.7 seconds of speech, and faster-whisper transcribed the test lines word for word.

I retired it in November. On an eleven-minute draft it couldn't hold attention, which is the only test that counts for narration.