Vocodex
Published: 7 July 2026
Vocodex is a voice assistant for working with code that runs entirely on my laptop. It hears me as I talk, reasons about the code I’m working on, and answers in a voice that belongs to nobody. There’s no cloud and no API calls, and no audio leaves the machine.
It started as a question: can this run on consumer hardware? It can, on a base M4 MacBook Pro with 16 GB of memory. I wrote the codec, the kernels and the voice from scratch.
How it works
Three models run in a chain. I reimplemented each one in Rust and checked it against its reference before it shipped.
- Hearing: Kyutai’s speech-to-text model, fed by my own port of the Mimi codec in 80 ms frames, so words appear while I’m still talking.
- Reasoning: Qwen3 4B Instruct, quantised to int4, at about 34 tokens a second.
- Speaking: Kyutai’s 1.6B text-to-speech model, with a voice I made for it.
All three run through about 35 hand-written Metal kernels, at conversational speed.
The voice
Kyutai’s text-to-speech model takes its voice from a tensor made by an encoder that was never released, so there’s no supported way to give it a new one. That tensor is just numbers, though, and numbers can be optimised. I froze the model and trained only the tensor against recordings, until it spoke in a voice that isn’t anyone’s.
Writing
- I taught a locked text-to-speech model to speak in a voice that doesn’t exist
- Rendering speech with 36 million parameters instead of 600 million, and it comes out clearer
- From 22.8 to 42.8 tokens a second without touching a matrix multiply