Once you get the hang of talking to a computer instead of typing, there is no going back. For a long while that went through Wispr Flow, which did the job well — accurate and fast, and its hands-free mode became muscle memory.

I have been itching to run a model locally, and this looked like a perfect candidate. Thus started the project of replacing Wispr Flow with a vibe-coded solution.

The constraints

My Mac is a 2019 Intel i9. No Apple Neural Engine, no M-series anything. Almost every on-device speech model shipped in the last two years assumes the opposite.

So before I wrote a line of app code, I recorded twenty utterances in my own voice: a Slack message, a code comment full of identifiers, jargon-heavy shell commands, a sixty-second ramble, and one clip of me saying nothing but “yes”.

That last one was not padding. Sub-second audio is exactly where one of the candidate models is documented to fall apart. Better to find that out on day zero than after building a whole app around it.

I benchmarked this with three local models — Parakeet TDT 0.6b v2, Whisper small.en and Moonshine base.en, all int8 — each on the same runtime, the same audio frontend, the same thread count — so the numbers compared models, not build systems.

Parakeet TDT 0.6b v2 won: 7.6% word error rate, 346ms median. Whisper small, the name everyone knows, scored 16.3% at over two seconds. Fine for a transcript you read later. Useless for text that should already be there when you stop talking.

That is the model running on my machine every time I speak to it now.

What it actually is

The whole thing is about 2,200 lines of Swift:

• A key press — F19, via an event tap.
• Audio capture — the engine is always running, keeping a 500ms rolling buffer, so the first syllable of a dictation is never clipped.
• Transcription — sherpa-onnx, an open-source speech runtime, through its C API, with the Parakeet weights, entirely offline.
• A vocabulary pass — a term list that fixes what the model has never seen.
• Insertion at the cursor — the text lands in whatever app has focus.

No Xcode. No CMake. No Python. Command Line Tools and a Makefile.

The boring parts were the hard parts

Getting speech to text was quick, and the model wrote most of it. What ate the time was everything around it.

Earning the key back. Hyper+Y triggered Wispr Flow, so it had to trigger mine. It broke immediately: my app never saw the key come up, because Hyper holds left-Option down underneath it and the release never registered as one. The fix was to stop being clever and remap to F19, an idle F-key with unambiguous down and up events.

Cancelling without pasting. Escape should abandon a dictation. But my app only listens to keys, it never swallows them, so Escape would reach the terminal behind it and interrupt whatever was running there. The fix is a key-blocking tap that exists only while a dictation is in flight, and is torn down the moment it ends.

Proper nouns. The worst sentence in my benchmark was wrong on every backend, up to 68% error — not acoustics, just words no model has seen. onnx came out as “Onx”. Vaani, the name for the project, came out as “Warne”. No model choice fixes that, so a term list rewrites them after transcription, whole-word matched so it cannot wreck a sentence it was not aimed at.

Dogfooding my own dictation app. Testing a change meant either killing the app mid-sentence or letting the test suite paste transcripts into whatever window had focus. So there are two copies now: production on F19, and a dev build on F18 with its own bundle id, which logs how many characters it transcribed instead of pasting them. They run side by side and neither sees the other’s keypress.

What is still wrong with it

The microphone indicator on my Mac is permanently lit.

That is the price of the rolling buffer: the audio engine never stops, so the first syllable is never lost. Stop it between dictations and I clip myself.

The cost is worse than an orange dot. That indicator is macOS’s one signal that something is listening, and my app keeps it lit around the clock — so it can never tell me a browser tab has started listening too. The signal is spent.

I measured the alternative — a cold microphone start costs about 195ms, a warm one 72 to 96ms — and it is close enough to be tempting. But the real dictations I logged put speech onset anywhere from 70ms to 1100ms after the key press, and I do not trust the 70ms number: it might be the sound of the key itself.

So it sits there, unresolved, in the app I use every day. But it is a compromise I willingly made for better accuracy.

What two days actually bought

The code was the cheap part. A model wrote most of it.

What took the time was deciding: recording a corpus before building anything, measuring three engines under one runtime, picking a key by how its events behave rather than how it feels, and being willing to leave a known flaw in place rather than trade away a privacy signal I have not finished thinking about.

Vibe coding made building it nearly free. It did not make knowing what to build free.