Whisper / Engineering case study
A transcription product with no server behind it
OpenAI's Whisper, running inside the visitor's own browser tab on their own graphics card. Audio is never uploaded, because there is nowhere to upload it to — the product ships as static files and every byte of computation happens on the client.
My role: sole engineer — architecture, model pipeline, design system, browser performance work. August 2026.
00 / Why
Free, because it costs nothing to run
Whisper is open source and free to use. That has not stopped a whole category of products from wrapping it in a login screen and charging a monthly fee — the model is someone else's work, the heavy lifting often happens on the customer's own hardware anyway, and the subscription is really for the wrapper around it.
This is the same capability, given away. It can be free because it genuinely costs nothing to operate: no server to pay for, no per-minute inference bill, no storage, no bandwidth beyond the model download — and that comes from a public CDN. The entire infrastructure is static file hosting. Charging a subscription for it would mean charging for something with no cost behind it.
Backend
None — static hosting only
Speed
5.7× faster than realtime
Models
3 sizes · 10 languages
Export
PDF, DOCX, TXT, SRT, VTT
01 / Architecture
Everything on the client
Three layers, none of them on a server. The privacy claim is not a policy — it is a consequence of there being no place for the audio to go.
React and TypeScript, sliced by feature rather than by file type — transcription and recording are separate modules, each owning its own model, hooks, worker and components. Design tokens were measured off a reference site's computed styles rather than eyeballed, so colour, radius and stroke values are exact rather than approximate.
Whisper runs through transformers.js on ONNX Runtime Web, inside a dedicated Web Worker so a long file never freezes the interface. WebGPU is the primary backend with a WebAssembly CPU fallback. Audio of any length is cut into 30-second windows with 5 seconds of overlap and stitched back together.
Model weights live in IndexedDB. Microphone capture uses MediaRecorder with a separate Web Audio graph feeding the waveform, so visualisation never touches the recording itself. Export to PDF and DOCX is generated in the browser and loaded on demand, keeping two megabytes of libraries out of the initial bundle.
02 / Decisions
Decisions that mattered
Five calls, each made after a measurement contradicted the obvious answer.
The largest model first looked broken: the download reached 100% and then hung with no error, forever. The cause was requesting an fp32 encoder for every model uniformly — for Large that is over a gigabyte of weights, past what WebGPU will allocate in a single buffer. Each model now has its own precision table. Large runs q4f16 at 538 MB instead of 1.5 GB; the smaller models keep a higher-precision encoder, because quantising the encoder is what costs accuracy and there is nothing to save at under 200 MB.
Why it matters: the same model went from never finishing to the fastest option in the product.
The library caches weights in Cache Storage, and the interface promised they would be reused. Inspecting the cache after a run showed the promise was false: the 185 MB decoder was stored, the 353 MB encoder was not. Chrome silently refuses to persist single entries past a few hundred megabytes, so the biggest file of every large model was re-downloaded on every single run. Swapping in an IndexedDB-backed cache removed the ceiling.
Why it matters: a repeat run of the largest model dropped from 143 seconds to 12, measured after a full page reload.
Beam search was exposed as a quality toggle, since it is what the reference Whisper implementation uses. Two things looked wrong in testing: the output was byte-identical to greedy decoding, and the "slower" mode was sometimes faster. Reading the library's generation loop explained both — it accepts a num_beams argument, computes the candidates, then keeps the first and breaks, with a TODO: Support beam search beside it.
Why it matters: the control was deleted rather than kept. A setting that does nothing is worse than no setting, because it spends the user's trust.
Every model was offered on both backends, which looked generous and was in fact a trap: the largest model on CPU would pull a 2.4 GB encoder and run several times slower than realtime. Benchmarking each pair produced a table of what genuinely works, and the interface now reflects it — Large is greyed out with the reason in place of its size when CPU is selected, and choosing CPU while it is active moves the selection down automatically.
Why it matters: a choice the product cannot honour is not a feature. Failing at the point of selection beats failing after a gigabyte of download.
The loading bar averaged the percentage of each file, so six tiny configuration files at 100% and one 353 MB file at 6% displayed as 87% complete. It now weights by bytes and prints the size of each file next to its progress. The recording timer was moved out of React entirely for a related reason: updating state ten times a second was re-rendering the dialog and tearing down any open dropdown inside it.
Why it matters: on a download this large, a progress bar that lies is the difference between waiting and giving up.
03 / Measurements
What the numbers actually say
19 seconds of speech, weights already cached. Realtime is audio length divided by compute time — higher is faster.
| Model | Download | WebGPU | CPU |
|---|---|---|---|
| Base | 197 MB | 5.8× | 1.4× |
| Small | 391 MB | 4.9× | 0.4× |
| Large v3 Turbo | 538 MB | 5.7× | unavailable |
The result that reorders everything: Large is both the most accurate and the fastest. Its decoder is only four layers deep, and the decoder is the part that runs sequentially, one token at a time — a bigger encoder costs one pass per window, a deeper decoder costs every token.
04 / Shipped
What is in the product
Recording defaults were changed too: the browser's echo cancellation and noise suppression are tuned for phone calls and reshape the spectrum the model was trained on, so both are switched off and the bitrate is raised.