Win Dictation: under the hood

The sections below back up every part of that judgement with specifics from the source.

Deliberately lean. No UI framework, no managed runtime, no garbage collector — just the OS and three libraries.

The whole product is native code with no framework abstraction between it and the Win32 API. That's the source of both its biggest strength (a tiny, fast, dependency-light binary) and its biggest cost (everything is hand-rolled, including the widgets).

A linear pipeline with a single inference step. Audio in, text out, no streaming loop.

Two side channels run alongside the main pipeline: a progress estimator that predicts and smooths the transcription countdown, and a timing model that learns this machine's speed and feeds back into the next prediction. Results return to the UI thread exclusively via PostMessage; shared flags are std::atomic.

The single most important thing to understand: this app was rebuilt from a live-streaming design into a push-to-talk batch design — and that was the right call.

An earlier version used the classic whisper.cpp streaming approach: a rolling 5–6 second window re-transcribed every ~0.4 seconds. That technique assumes spare cores. On the target machine — an Intel i5-7th-gen with two physical cores — the buffer backlogged, audio was re-transcribed, and the UI starved. The symptom looked like "the model is slow"; the real cause was an architecture that needed hardware the target didn't have.

The fix wasn't a faster model. It was removing the streaming loop entirely:

This is textbook root-cause engineering: the team correctly diagnosed that the bottleneck was the shape of the work, not its size, and changed the shape. Everything else in the codebase — the batch worker, the progress estimator, the physical-core thread default — follows from this one decision.

Four threads, one rule: only the UI thread touches the UI. Everything else reports back by message.

Cross-thread state is handled with discipline rather than locks where possible: std::atomic booleans (m_recording, m_busy, m_abort, g_modelLoaded, g_modelOk) gate state transitions, and the only shared buffer — the captured PCM — is protected by a dedicated mutex. The audio callback (driven by SDL's own thread) appends under that mutex; stop_and_transcribe swaps the buffer out under the same lock before handing it to the worker. That swap-not-copy handoff is a nice touch.

// transcriber.cpp — physical cores, not logical, by design
int Transcriber::default_threads() {
    unsigned hc = std::thread::hardware_concurrency();
    if (hc <= 2) return (int)std::max(1u, hc);
    return (int)(hc / 2);   // 4 logical → 2 worker threads
}

Defaulting to physical cores rather than hardware_concurrency() is the correct choice for compute-bound SIMD inference — hyperthreads contend for the same execution units and would only add scheduling overhead.

The app is small and mostly header-only outside the two big translation units. Here's where everything lives.

The decision to make most subsystems header-only and independent (timing.h, stats.h, history.h, downloader.h, settings.h) is a good one for a project this size: each is cohesive, individually readable, and free of cross-dependencies. The contrast with main.cpp — which absorbs everything else — is stark.

There are no buttons. Everything you see is painted onto one double-buffered surface — and that's a deliberate fix, not a shortcut.

The previous UI composited around nine separate themed child windows (buttons, statics, an edit). That produced thin hairline seams around every control — the hard-edged holes WS_CLIPCHILDREN punches per child, plus the edit's themed border. Rather than chase pixel borders, the rebuild eliminated the cause: collapse the controls into one painted region.

The model is a small immediate-mode system:

• A flat Widget g_w[] array — each entry is a kind, a rectangle, and hover/pressed/anim state. No HWNDs.
• LayoutWidgets() positions them; PaintSurface() draws each into an off-screen DC with GDI+, then blits once (no flicker).
• HitTest() maps a click point to a widget; OnClick() dispatches the action.
• An animation clock eases each widget's anim toward a target (hover 0.6, active 1.0) on a 16 ms timer that stops itself when nothing is moving — no idle CPU burn.

Two "views" — Main and Settings — render onto the same surface, toggled by SwitchView(). Dropdowns (mic, history) are the one exception: they're real top-level WS_POPUP windows, because a surface-painted dropdown would render behind the transcript edit (a child HWND always paints above its parent's surface). That's a correct, well-reasoned exception.

The one real child window that survives is the transcript EDIT — kept because a hand-rolled text editor with selection, scrolling, IME and undo is genuinely not worth rebuilding. Pragmatic.

The cost of this approach

Custom-painted controls are invisible to screen readers and UI Automation, and the app explicitly hides focus rectangles. The transcript box is accessible; the buttons are not. For a personal productivity tool this is a defensible trade, but it's the kind of thing worth stating out loud.

The crown jewel. Most apps fake a progress bar; this one runs a small statistical model that learns your machine.

Whisper only reports coarse progress (per 30-second chunk), so a naive bar jumps — 0, 34, 72, 100 — and a naive "time left" computed from stale percentages actually counts up. timing.h solves both. It has two parts.

1 — A learned timing model

Processing time is modelled as a linear function of audio length, proc = a + b·audio, fitted by decayed online least-squares. Each completed transcription feeds back a real sample; older samples decay (factor 0.97) so the model tracks the current machine state. Defaults are seeded per model family (tiny/base/small) so even the very first clip has a sane estimate, and the accumulators persist per-model in the INI.

void add_sample(double audio_sec, double proc_sec) {
    const double decay = 0.97;
    n*=decay; sx*=decay; sy*=decay; sxx*=decay; sxy*=decay;
    n+=1; sx+=audio_sec; sy+=proc_sec;
    sxx+=audio_sec*audio_sec; sxy+=audio_sec*proc_sec;
    recompute();   // closed-form slope/intercept
}

2 — A live estimator that only counts down

On stop, begin(predict) seeds a predicted total. As Whisper reports chunk progress, on_whisper() folds it in as a measurement via an EMA (α = 0.5) — nudging the estimate without the jumpy jumps. Meanwhile tick() advances a displayed "remaining" value that is strictly monotonic downward, with a clamped catch-up rate so it can speed up but never lurch backward, easing to 95% and snapping to 100% only on the real result.

void tick(double dt, float& out_frac, float& out_remaining) {
    t += dt; disp_rem -= dt;
    double raw_rem = std::max(0.0, T_hat - t);
    double err = raw_rem - disp_rem;
    if (err < 0) disp_rem += std::max(err, -maxCatchUp*dt);  // catch up, never jump back
    double frac = t/(t+disp_rem);
    if (frac > 0.95) frac = 0.95;   // park at 95% until done
    out_frac = (float)frac; out_remaining = (float)disp_rem;
}

This is more thought than most commercial apps put into a progress bar, and the test suite even asserts the progress is non-decreasing. It's the clearest signal in the codebase that someone cared about the feel of the product, not just its function.

The cleanest class in the project — a tidy boundary between the OS/model and the rest of the app.

Transcriber owns the Whisper context and the SDL device, and exposes a small, sensible surface: preload, reload, start_recording, stop_and_transcribe, cancel, plus state queries and two callbacks (result, progress). Inference parameters are configured sensibly for dictation — greedy sampling, no timestamps, no prior context, blank/non-speech suppression, temperature 0 — and an abort_callback lets a long transcription be cancelled mid-flight.

Two small details worth calling out:

• Silence trimming. Before inference, leading/trailing silence is trimmed from the clip — cheaper and more accurate than transcribing dead air. (Note: this trims the buffer; it does not auto-stop recording — see the doc-drift note in §13.)
• Output cleanup. clean_text() strips Whisper's [BLANK_AUDIO] / [NOISE] artifacts and trims whitespace, so the user never sees model noise.

The class is also defensively coded: a missing model file makes preload return false cleanly (the test suite verifies this), start_recording bails if a device won't open, and clips under ~0.3 s short-circuit to an empty result rather than invoking the model.

No database, no registry sprawl — everything is a file next to the executable. Transparent and portable.

Session history

Each session is one UTF-8 text file in history\, named by timestamp. The clever bit is live archiving: the first clip of a session creates the file; subsequent clips rewrite the same file with the full text. So a session is always one tidy, crash-safe file — not a scatter of fragments — and it appears in the History popup immediately. A g_sessionPath global plus a FinalizeSession() helper handle the edge cases (manual edits, typed-only sessions, loading an old entry without resurrecting it). The list is capped at 100 with automatic pruning.

Model downloads

The downloader is more robust than it needed to be, in a good way: it streams to a .part file then does an atomic rename on success (no half-files), honors the system proxy, follows the Hugging Face → CDN redirects, supports cancellation, allows only one download at a time, and sweeps up stray .part files at startup.

Settings, timing & stats

All three live in a single win-dictation.ini under different sections — app settings, per-model timing accumulators, and lifetime stats. Using the OS's own GetPrivateProfile* API means zero parsing code and a file a user can read and edit by hand. For an app of this scope, that's exactly the right level of machinery.

Following a single clip end-to-end ties the whole system together.

1. WM_HOTKEY fires → the app records g_prevForeground and the current selection (EM_GETSEL) so it knows where to paste and where to insert.
2. g_tx.start_recording() opens the SDL device; the audio callback appends PCM under the capture mutex and updates the RMS energy meter.
3. The 50 ms UI timer animates the level meter and ticks the on-screen recording clock.
4. Second WM_HOTKEY → g_est.begin(g_timing.predict(len)) seeds the progress estimate; g_tx.stop_and_transcribe() swaps the buffer to a worker thread.
5. The worker runs run_inference() → whisper_full. Whisper's progress callback posts WM_APP_PROGRESS; the estimator's on_whisper() EMA-folds it in.
6. On completion the worker posts WM_APP_RESULT.
7. The UI thread then, in order: snaps progress to 100%, records a real timing sample (add_sample + SaveTiming), updates lifetime stats, inserts the text at the saved caret with smart spacing via EM_REPLACESEL (undoable), archives the session, copies to the clipboard, and pastes into g_prevForeground.

Every piece of the architecture shows up in that one trip: the atomics, the message hand-back, the estimator, the learned timing feedback loop, the editable transcript, the live history. It's a coherent design.

All fixable, and none of it affects the running app. But it's exactly what a newcomer trips over.

If the next session had a short to-do list, this would be it — ordered by payoff for effort.

1. Purge or archive the stale docs high

   Delete or clearly mark CUDA-SETUP.md, QUICK-REBUILD-GPU.md, FIXES-APPLIED.md, TESTING.md and DESIGN.md as describing the retired streaming design. This is the single biggest improvement to onboarding, and it's nearly free.

2. Reconcile the README with reality high

   Remove the "500 ms silence auto-end" claim (or implement it). Update the model table and build commands to match the CPU-only product.

3. Delete the vestigial child windows medium

   Remove the hidden owner-draw controls and their dead WM_COMMAND handlers so there's exactly one code path per action. De-duplicate Copy.

4. Break up main.cpp medium

   Lift the Settings view, the popup window, and the GDI+ drawing helpers into their own files. Even a mechanical split dramatically improves navigability.

5. Fix the build standard & align build.ps1 medium

   Set CMAKE_CXX_STANDARD 17. Strip the CUDA detection from the build script and default it to fetching tiny.en.

6. Add an in-app hotkey picker low

   The Settings surface already exists; surfacing the hotkey there closes an obvious UX gap and removes a troubleshooting step.