Files
win-dictate/Architechture-and-dev-guide.md
T

412 lines
21 KiB
Markdown
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
# Win Dictation — Architecture Analysis & Dev Guide (Build 4)
Two parts:
- **Part A — Architecture analysis & recommendations.** Where the app stands now (post-fixes) and what's worth hardening.
- **Part B — Dev guide** for the two things you asked for: (1) removing the leftover Windows chrome (combo dropdown buttons, button hairlines, the line around *Pinned*), and (2) showing real transcription progress + a time estimate instead of a static "Transcribing…".
Context: native C++ / Win32 + SDL2 + whisper.cpp, CPU-only (i5-7th-gen, 2c/4t). Companion docs: `MODERN-UI-AND-FIXES.md`, `FINDINGS-FIXES-TESTS.md`.
---
# Part A — Architecture Analysis & Recommendations
## A1. Current architecture (it's in good shape)
```
┌── UI thread (message loop) ───────────────┐ ┌── Audio thread (SDL callback) ──┐
│ • window, hotkeys, tray, GDI+ paint │ │ • append f32 samples to │
│ • owns Transcriber │◀────│ m_capture (mutex) │
│ • marshals worker results via PostMessage│ │ • update VU energy (atomic) │
└──────────────┬────────────────────────────┘ └─────────────────────────────────┘
│ stop_and_transcribe()
┌── Worker thread (one per utterance) ──────┐
│ • run_inference(): whisper_full() ONCE │
│ • PostMessage(WM_APP_RESULT, text) │
└────────────────────────────────────────────┘
```
State machine: **Idle → Recording → Transcribing → Idle**, with the model preloaded on a detached thread at startup.
**What's genuinely good now:**
- **Clean thread boundaries.** Audio capture, inference, and UI never block each other; all cross-thread hand-offs are atomics or `PostMessage`. This is the right shape.
- **Batch (record-then-transcribe).** The correct model for dictation on a slow CPU — no real-time deadline, latency independent of clip length.
- **Core is decoupled and unit-testable** (`Transcriber` + `run_inference`/`transcribe_sync`, `text_util.h`). The UI is a thin shell over it.
- **Robust startup** (single-instance, honest `g_modelOk`, `g_initializing` guard, `m_cfg_mtx`) — the earlier crashes are designed out, not patched over.
The remaining issues are **polish and hardening**, not structural. No rewrite needed.
## A2. Recommendations (ranked by impact ÷ effort)
| # | Recommendation | Why | Effort |
|---|---|---|---|
| 1 | **Transcription progress + ETA** (Part B2) | The #1 UX gap: a 1:40 clip shows a static "Transcribing…" with no sign of life. whisper.cpp already exposes a progress callback — wire it up. | S |
| 2 | **De-theme the stock controls** (Part B1.1) | The hairlines around the buttons / *Pinned* are themed-edge leftovers. `SetWindowTheme(h, L"", L"")` + `WS_CLIPCHILDREN` + hide-focus removes them. | S |
| 3 | **Replace the comboboxes with custom dropdowns** (Part B1.2) | A `CBS_OWNERDRAWFIXED` combo *cannot* hide its native dropdown button — that's the "second arrow" you see. The only clean fix is to not use a combobox. | M |
| 4 | **Persist settings** | Mic, model, pin, auto-paste, window position all reset every launch. A tiny INI (`WritePrivateProfileStringW`) or registry blob fixes it. | S |
| 5 | **Cancel/abort during transcription** | Pairs with #1 — if a long clip is wrong, let the user abort via `whisper_full_params.abort_callback`. | S |
| 6 | **Delete the stale `src/CMakeLists.txt`** | It references removed APIs (`init`, `is_using_gpu`) and a `test-audio` target the root build ignores. It's a trap for the next contributor. | XS |
| 7 | **Bound the capture buffer** | `m_capture` grows ~1.9 MB/30 s with no cap. Add a soft limit (e.g. auto-stop at 10 min) so a forgotten recording can't grow unbounded. | XS |
| 8 | **Configurable hotkeys + conflict check** | `Ctrl+Shift+Space`/`H` are hard-coded; `RegisterHotKey` failures are silently ignored. Surface a warning and allow remap. | M |
| 9 | **Lightweight logging** | A single rotating log line per session (model, threads, last error) makes field issues diagnosable without a debugger. | S |
**Architectural note for the future:** the app leans on owner-drawing *stock* controls (buttons, combos). That's fine at this size, but every stock control fights you on chrome (edges, focus cues, dropdown buttons). If the UI grows, the higher-leverage move is a small set of **fully custom controls** (a `Button`, a `Select`, a `ProgressBar` you paint entirely in `WM_PAINT` and hit-test yourself) rather than more owner-draw patches. Part B1.2's custom dropdown is the first step down that path.
---
# Part B — Dev Guide
## B1. Remove the leftover Windows chrome
There are **three** separate sources of "Windows-y" pixels, each with a different cause:
| What you see | Cause |
|---|---|
| Thin light line around *Pinned*; vertical line left + horizontal line top of Copy/Paste/Clear | The visual-style **themed edge** drawn behind owner-draw buttons, plus **focus rectangles** |
| Two arrows / a boxy button on the mic + model selectors | A `CBS_OWNERDRAWFIXED` combo still lets the **system draw the dropdown button** and field border — owner-draw only covers the text/items |
### B1.1 Kill the button hairlines + focus rectangles (quick, high-impact)
Three changes, all small:
**(a) De-theme the owner-draw controls.** `SetWindowTheme(h, L"", L"")` strips the UxTheme styling from a control so Windows stops painting its themed border/edge behind your `WM_DRAWITEM`. Add the header/lib and call it on every owner-draw control right after you create it:
```cpp
#include <uxtheme.h>
#pragma comment(lib, "uxtheme.lib")
// after creating each owner-draw button (Record, Pin, Copy, Paste, Clear):
SetWindowTheme(hBtn, L"", L""); // remove themed edges
SetWindowSubclass(hBtn, BtnProc, 1, 0);
```
**(b) Stop the parent painting under the children.** Add `WS_CLIPCHILDREN` to the main window so its double-buffered `WM_PAINT` never bleeds a pixel into a child's rectangle:
```cpp
hMainWnd = CreateWindowExW(
WS_EX_TOPMOST,
L"WhisperDictationClass", L"Dictation",
WS_OVERLAPPEDWINDOW | WS_CLIPCHILDREN, // <- add WS_CLIPCHILDREN
CW_USEDEFAULT, CW_USEDEFAULT, 400, 340,
nullptr, nullptr, hInstance, nullptr);
```
**(c) Hide focus rectangles app-wide.** The dotted/thin rect that appears on whichever control has keyboard focus (often *Pinned* after you click it). Tell the UI-state machine to keep focus cues hidden — once, after the controls exist:
```cpp
SendMessageW(hMainWnd, WM_CHANGEUISTATE, MAKEWPARAM(UIS_SET, UISF_HIDEFOCUS), 0);
```
After (a)+(b)+(c), the only remaining stock chrome is the combo dropdown button, handled next.
### B1.2 Replace the comboboxes with a custom dropdown (removes the native button)
A combobox always owns its dropdown button — you can't draw it away. Replace each combo with a **flat owner-draw "select" button** (you already draw exactly this look in `DrawCombo`) that opens a **custom dark popup window**. No native button, no field edge, fully on-theme.
**Data model** (replace `g_modelComboPaths` usage as needed; keep a label list + selection per selector):
```cpp
std::vector<std::wstring> g_audioItems; int g_audioSel = 0;
std::vector<std::wstring> g_modelItems; int g_modelSel = 0; // parallel to g_modelComboPaths
```
Populate these in `RefreshAudioDevices` / `RefreshModelList` instead of `CB_ADDSTRING`. Create `ID_SEL_AUDIO` / `ID_SEL_MODEL` as `BS_OWNERDRAW` buttons (not comboboxes) and draw them with a field renderer:
```cpp
void DrawSelect(LPDRAWITEMSTRUCT d, const std::wstring& text) {
Graphics g(d->hDC); g.SetSmoothingMode(SmoothingModeAntiAlias);
Rect rc(d->rcItem.left, d->rcItem.top, d->rcItem.right-d->rcItem.left, d->rcItem.bottom-d->rcItem.top);
SolidBrush bg(C_BG); g.FillRectangle(&bg, rc);
Rect field = rc; field.Inflate(-1,-1);
bool hover = GetWindowLongPtr(d->hwndItem, GWLP_USERDATA) != 0;
FillRound(g, hover ? C_SURFACEHI : C_SURFACE, field, 9);
StrokeRound(g, C_BORDER, field, 9, 1.0f);
Font f(d->hDC, g_fUI);
RectF tb((REAL)field.X+10, (REAL)field.Y, (REAL)(field.Width-28), (REAL)field.Height);
DrawTextC(g, text.c_str(), f, C_TEXT, tb, StringAlignmentNear, StringAlignmentCenter);
int cx = field.GetRight()-16, cy = field.Y + field.Height/2; // our chevron, the only one now
Pen pen(C_TEXTDIM, 1.6f);
g.DrawLine(&pen, cx-4, cy-2, cx, cy+2); g.DrawLine(&pen, cx, cy+2, cx+4, cy-2);
}
// in WM_DRAWITEM:
// case ID_SEL_AUDIO: DrawSelect(d, g_audioItems.empty()?L"No devices":g_audioItems[g_audioSel]); return TRUE;
// case ID_SEL_MODEL: DrawSelect(d, g_modelItems.empty()?L"—":g_modelItems[g_modelSel]); return TRUE;
```
**The popup window** — a small borderless top-level you paint yourself; closes on pick or focus loss:
```cpp
struct PopupState { std::vector<std::wstring> items; int sel; int hot; HWND owner; int ctrlId; };
static PopupState g_pop;
LRESULT CALLBACK PopupProc(HWND h, UINT m, WPARAM w, LPARAM l) {
switch (m) {
case WM_MOUSEMOVE: {
int row = GET_Y_LPARAM(l) / 30;
if (row != g_pop.hot) { g_pop.hot = row; InvalidateRect(h, nullptr, FALSE); }
TRACKMOUSEEVENT t{ sizeof(t), TME_LEAVE, h, 0 }; TrackMouseEvent(&t);
return 0;
}
case WM_MOUSELEAVE: g_pop.hot = -1; InvalidateRect(h, nullptr, FALSE); return 0;
case WM_LBUTTONUP: {
int row = GET_Y_LPARAM(l) / 30;
if (row >= 0 && row < (int)g_pop.items.size())
PostMessageW(g_pop.owner, WM_APP_SELECT, (WPARAM)g_pop.ctrlId, (LPARAM)row);
DestroyWindow(h); return 0;
}
case WM_KILLFOCUS: DestroyWindow(h); return 0;
case WM_ERASEBKGND: return 1;
case WM_PAINT: {
PAINTSTRUCT ps; HDC hdc = BeginPaint(h, &ps);
RECT rc; GetClientRect(h, &rc);
HDC mem = CreateCompatibleDC(hdc);
HBITMAP bmp = CreateCompatibleBitmap(hdc, rc.right, rc.bottom);
HBITMAP old = (HBITMAP)SelectObject(mem, bmp);
{
Graphics g(mem); g.SetSmoothingMode(SmoothingModeAntiAlias);
Rect all(0,0,rc.right,rc.bottom);
FillRound(g, C_SURFACE, all, 10); StrokeRound(g, C_BORDER, all, 10, 1.0f);
Font f(mem, g_fUI);
for (int i = 0; i < (int)g_pop.items.size(); ++i) {
Rect row(3, i*30+3, rc.right-6, 28);
if (i == g_pop.hot) FillRound(g, C_SURFACEHI, row, 7);
RectF tb((REAL)row.X+9,(REAL)row.Y,(REAL)row.Width-12,(REAL)row.Height);
DrawTextC(g, g_pop.items[i].c_str(), f, (i==g_pop.sel)?C_ACCENT:C_TEXT,
tb, StringAlignmentNear, StringAlignmentCenter);
}
}
BitBlt(hdc,0,0,rc.right,rc.bottom,mem,0,0,SRCCOPY);
SelectObject(mem, old); DeleteObject(bmp); DeleteDC(mem);
EndPaint(h, &ps); return 0;
}
}
return DefWindowProc(h, m, w, l);
}
void ShowSelectPopup(HWND owner, int ctrlId, const std::vector<std::wstring>& items, int sel) {
static bool reg = false;
if (!reg) { WNDCLASSEXW wc{ sizeof(wc) }; wc.lpfnWndProc = PopupProc; wc.hInstance = hInst;
wc.hCursor = LoadCursor(nullptr, IDC_ARROW); wc.lpszClassName = L"DictPopup";
RegisterClassExW(&wc); reg = true; }
g_pop = { items, sel, -1, owner, ctrlId };
RECT rc; GetWindowRect(GetDlgItem(owner, ctrlId), &rc);
int h = (int)items.size()*30 + 6, wdt = rc.right - rc.left;
HWND p = CreateWindowExW(WS_EX_TOOLWINDOW | WS_EX_TOPMOST, L"DictPopup", L"",
WS_POPUP, rc.left, rc.bottom+2, wdt, h, owner, nullptr, hInst, nullptr);
int corner = DWMWCP_ROUND; DwmSetWindowAttribute(p, DWMWA_WINDOW_CORNER_PREFERENCE, &corner, sizeof(corner));
ShowWindow(p, SW_SHOWNA); SetForegroundWindow(p); SetFocus(p);
}
```
Wire it up: clicking a selector opens the popup; the popup posts the chosen row back:
```cpp
// WM_COMMAND:
case ID_SEL_AUDIO: ShowSelectPopup(hWnd, ID_SEL_AUDIO, g_audioItems, g_audioSel); break;
case ID_SEL_MODEL: ShowSelectPopup(hWnd, ID_SEL_MODEL, g_modelItems, g_modelSel); break;
// new message handler:
case WM_APP_SELECT: { // #define WM_APP_SELECT (WM_USER + 5)
int ctrlId = (int)wParam, idx = (int)lParam;
if (ctrlId == ID_SEL_AUDIO) { g_audioSel = idx; g_config.capture_id = idx; }
else if (ctrlId == ID_SEL_MODEL && idx < (int)g_modelComboPaths.size()) {
g_modelSel = idx;
g_config.model_path = exe_dir() + "\\" + g_modelComboPaths[idx];
g_modelLoaded = false; g_modelOk = false; SetStatus(hWnd, L"Loading model…");
std::thread([]{ bool ok=g_tx.reload(g_config); g_modelOk=ok; g_modelLoaded=true; }).detach();
}
InvalidateRect(GetDlgItem(hWnd, ctrlId), nullptr, FALSE);
} break;
```
You can now delete the `CBS_*`/`WM_MEASUREITEM`/`DrawCombo` combo code and the `WM_CTLCOLORLISTBOX` handler. The selectors are now pixel-identical to your other controls with exactly one (your) chevron.
> Lighter alternative if you don't want a popup window yet: keep the combos but call `SetWindowTheme(hCombo, L"", L"")` — it flattens the dropdown button to a plain square. It's *less* boxy but the button is still there, so the popup-window route above is the real fix.
---
## B2. Transcription progress + time estimate
You don't have to guess the timing — **whisper.cpp reports real progress.** `whisper_full_params` has a `progress_callback` that fires repeatedly during inference with an `int` 0100 (fraction of the audio processed). Feed that to a determinate progress bar + a live ETA, so "Transcribing…" becomes "Transcribing 1:40 of audio · 45% · ~9s left".
### B2.1 transcriber — expose progress + audio length
`transcriber.h` (public):
```cpp
using ProgressCb = std::function<void(int)>; // 0..100
void set_progress_callback(ProgressCb cb) { m_on_progress = std::move(cb); }
float audio_seconds() const { return m_audio_seconds.load(); }
void request_cancel() { m_abort = true; } // optional (B2.4)
```
`transcriber.h` (private):
```cpp
ProgressCb m_on_progress;
std::atomic<float> m_audio_seconds{0.0f};
std::atomic<bool> m_abort{false};
static void s_progress(struct whisper_context*, struct whisper_state*, int p, void* ud);
static bool s_abort(void* ud);
```
`transcriber.cpp` — the static trampolines and the `run_inference` hook:
```cpp
void Transcriber::s_progress(whisper_context*, whisper_state*, int p, void* ud) {
auto* self = static_cast<Transcriber*>(ud);
if (self && self->m_on_progress) self->m_on_progress(p);
}
bool Transcriber::s_abort(void* ud) {
auto* self = static_cast<Transcriber*>(ud);
return self && self->m_abort.load();
}
std::string Transcriber::run_inference(std::vector<float>& audio) {
if (!m_ctx) return "";
m_abort = false;
if (m_cfg.trim_silence) trim_silence(audio);
m_audio_seconds = (float)(audio.size() / (double)WHISPER_SAMPLE_RATE); // for the UI
whisper_full_params wp = whisper_full_default_params(WHISPER_SAMPLING_GREEDY);
wp.print_progress = false; wp.print_realtime = false; wp.print_timestamps = false;
wp.no_timestamps = true; wp.translate = false;
wp.language = m_cfg.language.c_str();
wp.n_threads = m_cfg.n_threads;
wp.no_context = true; wp.suppress_blank = true; wp.suppress_nst = true; wp.temperature = 0.0f;
wp.progress_callback = &Transcriber::s_progress; // <- live progress
wp.progress_callback_user_data = this;
wp.abort_callback = &Transcriber::s_abort; // <- optional cancel
wp.abort_callback_user_data = this;
std::string out;
if (whisper_full(m_ctx, wp, audio.data(), (int)audio.size()) == 0) {
int n = whisper_full_n_segments(m_ctx);
for (int i = 0; i < n; ++i) { const char* t = whisper_full_get_segment_text(m_ctx, i); if (t) out += t; }
out = clean_text(out);
}
return out;
}
```
> If your `whisper.h` predates `abort_callback`, drop those two lines — `progress_callback` has been in whisper.cpp far longer and is what matters here.
### B2.2 main.cpp — progress state + ETA
```cpp
#define WM_APP_PROGRESS (WM_USER + 6)
std::atomic<int> g_progress{0};
DWORD g_busyStart = 0;
// at startup, next to set_result_callback:
g_tx.set_progress_callback([](int p){ PostMessage(hMainWnd, WM_APP_PROGRESS, (WPARAM)p, 0); });
```
Start the clock when transcription begins (HK_TOGGLE stop branch):
```cpp
} else { // was recording -> stop
g_busyStart = GetTickCount();
g_progress = 0;
g_tx.stop_and_transcribe();
SetStatus(hWnd, L"Transcribing…");
}
```
Receive progress and repaint the bar:
```cpp
case WM_APP_PROGRESS:
g_progress = (int)wParam;
InvalidateRect(hWnd, &g_vuRect, FALSE);
return 0;
```
ETA text (replace the `is_busy()` branch in `UpdateStatus`):
```cpp
} else if (g_tx.is_busy()) {
int p = g_progress.load();
float total = g_tx.audio_seconds();
float elapsed = (GetTickCount() - g_busyStart) / 1000.0f;
int mm = (int)total / 60, ss = (int)total % 60;
if (p >= 3) {
float est = elapsed * 100.0f / p; // projected total
float remain = est - elapsed; if (remain < 0) remain = 0;
swprintf_s(buf, L"Transcribing %d:%02d • %d%% • ~%ds left", mm, ss, p, (int)(remain + 0.5f));
} else {
swprintf_s(buf, L"Transcribing %d:%02d of audio…", mm, ss);
}
SetStatus(hwnd, buf);
}
```
### B2.3 Reuse the VU strip as a determinate progress bar
While recording the strip shows the VU; while transcribing it shows progress. Add a renderer and branch in `WM_PAINT`:
```cpp
void DrawProgress(Graphics& g, const RECT& r, float frac) {
Rect track(r.left, r.top, r.right - r.left, r.bottom - r.top);
FillRound(g, C_SURFACEHI, track, 4);
frac = frac < 0 ? 0 : (frac > 1 ? 1 : frac);
int w = (int)((r.right - r.left) * frac);
if (w > 0) { Rect fill(r.left, r.top, w, r.bottom - r.top); FillRound(g, C_ACCENT, fill, 4); }
}
```
```cpp
// in WM_PAINT, where you currently call DrawVU:
if (g_tx.is_busy()) DrawProgress(g, g_vuRect, g_progress.load() / 100.0f);
else DrawVU(g, g_vuRect, g_energy);
```
Keep it ticking between callbacks so the ETA counts down smoothly — in `WM_TIMER`, add:
```cpp
else if (g_tx.is_busy()) {
InvalidateRect(hWnd, &g_vuRect, FALSE); // (UpdateStatus already runs each tick below)
}
```
That's it: a moving bar, a percentage, and a shrinking "~Ns left" — the user can see it's alive and roughly how long is left. The estimate self-corrects as real progress arrives (the first few percent are rougher; it tightens quickly).
### B2.4 (Optional) Cancel a long transcription
You added `request_cancel()` + the abort callback in B2.1. Hook it so that pressing the button **while busy** aborts instead of being ignored, and treat the empty result as "Cancelled":
```cpp
// top of the HK_TOGGLE handler:
if (g_tx.is_busy()) { g_tx.request_cancel(); SetStatus(hWnd, L"Cancelling…"); break; }
```
`whisper_full` returns promptly with no/aborted segments → your existing `WM_APP_RESULT` empty-string path shows "No speech detected"; change that label to "Cancelled" when a cancel was requested if you want to distinguish them.
---
## B3. Build & test
- **New link deps:** `uxtheme.lib` (added via `#pragma comment` in B1.1). GDI+ is already linked.
- **New files:** none required for B2; B1.2 adds the popup window proc inside `main.cpp` (no new translation unit).
- **Rebuild:** `cmake --build build --config Release` as usual.
**Verify:**
- [ ] No hairline around *Pinned*; no left/top lines on Copy/Paste/Clear; focus no longer draws a dotted rect.
- [ ] Mic + model selectors show a single (your) chevron, open a dark rounded popup, and selecting reloads the model / switches device.
- [ ] Record a ~12 min clip → the strip fills as a progress bar, the status shows `%` and a shrinking `~Ns left`, and it completes (no static "Transcribing…").
- [ ] (If added) pressing the button mid-transcription cancels promptly.
**Headless regression (extends `tests/test_core.cpp` from `FINDINGS-FIXES-TESTS.md`):**
- Set a progress callback that records the max value seen; assert it reaches ~100 for `samples/jfk.wav`, and that callbacks arrive in non-decreasing order. This locks in that progress reporting keeps working across whisper.cpp upgrades.
```cpp
int last = -1, maxp = 0; bool monotonic = true;
t.set_progress_callback([&](int p){ if (p < last) monotonic = false; last = p; if (p > maxp) maxp = p; });
t.transcribe_sync(audio);
CHECK(monotonic, "progress is non-decreasing");
CHECK(maxp >= 95, "progress reaches ~100%");
```
---
*Apply order: B1.1 (5 min, instant visual win) → B2 (progress, the big UX gain) → B1.2 (custom dropdowns) → optional B2.4 cancel. Each is independent and safe to ship on its own.*