Docs refresh: add user manual & architecture review; update README with accurate details
This commit is contained in:
Win Dictation Dev
@@ -0,0 +1,630 @@
|
||||
# Win Dictation — UI & Progress Rebuild Guide
|
||||
|
||||
A design-led plan to (1) kill the "thin lines" problem at its architectural root rather than patching it, and (2) replace the broken progress bar with a self-calibrating, smoothly-animated estimator that learns this machine's transcription speed and fuses whisper's own progress signal.
|
||||
|
||||
This is implementation guidance with concrete code. You build on Windows; nothing here is compiled or tested in place.
|
||||
|
||||
---
|
||||
|
||||
## Table of contents
|
||||
|
||||
1. [Part 1 — The interface](#part-1--the-interface)
|
||||
- [1.1 Why the lines are really there](#11-why-the-lines-are-really-there)
|
||||
- [1.2 The architectural fix: one surface](#12-the-architectural-fix-one-surface)
|
||||
- [1.3 Two tiers: GDI+ vs Direct2D](#13-two-tiers-gdi-vs-direct2d)
|
||||
- [1.4 A real design language](#14-a-real-design-language)
|
||||
- [1.5 Component specs](#15-component-specs)
|
||||
- [1.6 Rendering scaffold + hit-testing (code)](#16-rendering-scaffold--hit-testing-code)
|
||||
- [1.7 The transcript field & DPI](#17-the-transcript-field--dpi)
|
||||
- [1.8 Migration order from today's main.cpp](#18-migration-order-from-todays-maincpp)
|
||||
2. [Part 2 — The progress system](#part-2--the-progress-system)
|
||||
- [2.1 Why it's broken today](#21-why-its-broken-today)
|
||||
- [2.2 The plan: predict, then correct](#22-the-plan-predict-then-correct)
|
||||
- [2.3 Persistent per-model timing history](#23-persistent-per-model-timing-history)
|
||||
- [2.4 The live estimator (smooth countdown + fusion)](#24-the-live-estimator-smooth-countdown--fusion)
|
||||
- [2.5 `timing.h` — full code](#25-timingh--full-code)
|
||||
- [2.6 Wiring into main.cpp](#26-wiring-into-maincpp)
|
||||
- [2.7 Tuning & edge cases](#27-tuning--edge-cases)
|
||||
3. [Part 3 — Cleanup checklist](#part-3--cleanup-checklist)
|
||||
4. [Part 4 — Suggested build order](#part-4--suggested-build-order)
|
||||
|
||||
---
|
||||
|
||||
# Part 1 — The interface
|
||||
|
||||
## 1.1 Why the lines are really there
|
||||
|
||||
The hairlines aren't one bug; they're an emergent property of how the window is built. Today the UI is roughly **nine separate child windows** living on top of the main window:
|
||||
|
||||
- `BUTTON` (owner-draw): Record, Pin, Copy, Paste, Clear
|
||||
- `BUTTON` (owner-draw) used as selects: mic, model
|
||||
- `EDIT` (multiline): the transcript
|
||||
- `STATIC`: status + placeholder
|
||||
|
||||
Each child is its own HWND with its own device context, its own paint timing, and — because the parent uses `WS_CLIPCHILDREN` — its own **hard-clipped rectangle punched out of the parent's paint**. That single fact is the source of the lines:
|
||||
|
||||
1. **Seams at every child boundary.** The parent paints its background/panel, then Windows clips out each child rectangle and the child paints itself. The boundary between "parent pixels" and "child pixels" is a 1px hard edge. Any difference in rounding, antialiasing, or color across that edge reads as a hairline — even when both sides *intend* to be the same dark color.
|
||||
2. **Theme chrome you didn't ask for.** The `EDIT` control draws its own themed 1px border and a **light-mode scrollbar** (the pale bar on the right of your screenshot). `SetWindowTheme(h, L"", L"")` on the buttons disables visual styles but doesn't make seams go away.
|
||||
3. **Square corners around round shapes.** Your chips are drawn rounded, but the *child window* is rectangular, so the artifact rectangle has sharp corners that don't follow the chip — which is exactly what's visible around the Record pill and the Copy/Paste/Clear buttons.
|
||||
|
||||
So removing `StrokeRound(..., C_BORDER, ...)` only removes the *intentional* borders. The *structural* hairlines (items 1–3) remain. That's why it feels like a band-aid: **you can't fully remove seams while compositing many themed child windows.**
|
||||
|
||||
> **Root cause, one sentence:** the window is assembled from many separate themed/owner-draw child HWNDs, and the boundaries between them can never be made perfectly seamless. The fix is to stop having those boundaries.
|
||||
|
||||
## 1.2 The architectural fix: one surface
|
||||
|
||||
Render the **entire window as a single double-buffered surface**, immediate-mode:
|
||||
|
||||
- The parent's `WM_PAINT` draws *everything* — background, the card, every button, the selects, the status line, the VU/progress strip — onto **one off-screen bitmap**, then blits it once. (You already do this for the background and panel; we extend it to cover all chrome.)
|
||||
- **There are no child windows for chrome.** "Buttons" become **painted regions** described by a small data model (a rect + a kind + interaction state). There is exactly one surface, so there are zero inter-window seams. Antialiasing, radii, spacing, shadows, and animation are all under your control.
|
||||
- **Interaction** is handled in the parent: `WM_MOUSEMOVE` / `WM_LBUTTONDOWN` / `WM_LBUTTONUP` hit-test against the widget rects; you track hover/pressed/focus yourself and invalidate. (The window is tiny — invalidating the whole client area each frame is cheap.)
|
||||
- **The one exception is the transcript**, which stays a real `EDIT` child because you genuinely want selection, caret, scrolling, and IME. We make it *visually chrome-less* and inset it inside the painted card so the card is the only visible frame (see [1.7](#17-the-transcript-field--dpi)).
|
||||
|
||||
This is the same "retained data model + immediate-mode paint" approach used by every good custom-drawn desktop UI. Separation between elements comes from **fills, spacing, and elevation — not outlines.** Once outlines stop being load-bearing, the hairline problem is gone by construction.
|
||||
|
||||
## 1.3 Two tiers: GDI+ vs Direct2D
|
||||
|
||||
You said you'll happily take more effort for a result that looks genuinely good. Here are the two honest options.
|
||||
|
||||
### Tier 1 — GDI+ single-surface (recommended baseline)
|
||||
|
||||
- Keep GDI+ (already in the project). Move all drawing into one parent paint routine that renders to a 32-bit DIB back-buffer, then `BitBlt`.
|
||||
- Reuse your existing helpers (`FillRound`, `StrokeRound`, `DrawTextC`) — they're good. You're changing *what hosts them*, not the primitives.
|
||||
- Add an animation clock + hover/press state.
|
||||
- **Effort:** moderate. **Payoff:** the seams disappear, you get full control of spacing/elevation/motion, and it will look clean and modern. This removes 100% of the reported problem.
|
||||
- **Limitations:** GDI+ has no true GPU compositing; soft drop-shadows must be faked (pre-blurred bitmap or layered alpha), and very large blurs are slow. For a 400×340 utility this is a non-issue.
|
||||
|
||||
### Tier 2 — Direct2D + DirectWrite (premium path)
|
||||
|
||||
- GPU-accelerated geometry with flawless antialiasing, real `ID2D1Effect` drop shadows / Gaussian blur, per-primitive opacity layers, and **DirectWrite** text with subpixel positioning (noticeably crisper labels, especially at fractional DPI).
|
||||
- Pairs naturally with a swap-chain or a DC render target; integrates with DWM for tear-free animation at the monitor refresh rate.
|
||||
- Optionally add **Windows.UI.Composition / DirectComposition** for soft shadows and an acrylic/mica backdrop — a true Windows 11 feel.
|
||||
- **Effort:** higher (COM lifetimes, device-lost handling, more setup). **Payoff:** the highest visual ceiling and the best foundation if this app grows.
|
||||
- You can still keep the `EDIT` child for the transcript layered above the D2D surface.
|
||||
|
||||
**Recommendation:** Build **Tier 1 now** — it eliminates the actual defect and looks great, and almost all of the work (the design language, the widget model, the interaction layer, the progress system in Part 2) is *identical* regardless of renderer. If you later want the extra polish, swapping the draw calls to Direct2D is a contained change because the data model and layout stay the same. The rest of this guide is written renderer-agnostic with GDI+ code samples.
|
||||
|
||||
## 1.4 A real design language
|
||||
|
||||
The current look is "many bordered boxes." The target look is **one calm, elevated card** where hierarchy comes from type, spacing, and a single light source — not lines.
|
||||
|
||||
### Tokens (define once)
|
||||
|
||||
```cpp
|
||||
// ---- color tokens (ARGB) ----
|
||||
const Color T_BG (255, 0x0E, 0x10, 0x14); // app backdrop (near-black)
|
||||
const Color T_CARD (255, 0x16, 0x19, 0x20); // elevated card
|
||||
const Color T_CARD_HI (255, 0x1E, 0x22, 0x2B); // hovered surface
|
||||
const Color T_CARD_LO (255, 0x12, 0x15, 0x1B); // pressed surface / wells
|
||||
const Color T_TEXT (255, 0xEC, 0xEE, 0xF2); // primary text
|
||||
const Color T_DIM (255, 0x8A, 0x90, 0x9C); // secondary text
|
||||
const Color T_FAINT (255, 0x5A, 0x60, 0x6C); // tertiary / icons at rest
|
||||
const Color T_ACCENT (255, 0x6E, 0x8B, 0xFF); // primary action
|
||||
const Color T_ACCENT_HI (255, 0x83, 0x9C, 0xFF); // accent hover
|
||||
const Color T_DANGER (255, 0xFF, 0x5C, 0x5C); // recording
|
||||
const Color T_GOOD (255, 0x46, 0xD3, 0x9A); // level / success
|
||||
|
||||
// The ONLY "edge" allowed: a low-alpha top highlight on the card,
|
||||
// to read as "lit from above." Never a full gray rectangle.
|
||||
const Color T_TOPLIGHT (26, 0xFF, 0xFF, 0xFF); // ~10% white
|
||||
```
|
||||
|
||||
**Principle:** elements are distinguished by *fill* (`T_CARD` vs `T_CARD_HI`), by *space* (generous padding), and by *elevation* (the card sits on the backdrop, optionally with a soft shadow). Outlines are reserved for nothing, or at most one hairline-as-toplight on the card itself.
|
||||
|
||||
### Type scale (Segoe UI Variable, which you already load)
|
||||
|
||||
| Role | Size (logical px) | Weight | Color |
|
||||
|------|------|--------|-------|
|
||||
| Primary state ("Record" / "Stop" / "Transcribing") | 16 | SemiBold | white on accent / `T_TEXT` |
|
||||
| Body / transcript | 16 | Regular | `T_TEXT` |
|
||||
| Buttons (ghost) | 14 | Medium | `T_DIM` → `T_TEXT` on hover |
|
||||
| Status caption | 12.5 | Regular | `T_DIM` |
|
||||
| Micro (threads, %, ETA) | 11.5 | Regular | `T_FAINT` |
|
||||
|
||||
### Elevation & radius
|
||||
|
||||
- Card radius **16**; inner controls radius **10–11**; progress/level pill radius = half-height.
|
||||
- Optional soft shadow under the card (Tier 1: a pre-rendered blurred rounded-rect bitmap at ~22% alpha, offset y+6, blur ~18; Tier 2: a D2D shadow effect). Subtle — it should read as depth, not drama.
|
||||
|
||||
### Motion (this is what makes it feel "good", not just look good)
|
||||
|
||||
- Hover/press fills cross-fade over **120–160ms**, ease-out-cubic.
|
||||
- Recording state: a **1.2s sine "breathing"** on the record pill + a live waveform (see below).
|
||||
- Progress: bar width and the % label are **eased**, never snapped (except the final 100%).
|
||||
- Drive all of it from one animation clock (Section 1.6). Run the timer at ~16ms **only while something is animating**, and idle otherwise (don't burn CPU on a 2-core machine when nothing moves).
|
||||
|
||||
## 1.5 Component specs
|
||||
|
||||
**Record (hero).** Full-width pill, `T_ACCENT` fill, white glyph + label. States:
|
||||
- *Idle:* circle glyph + "Record". Hover → `T_ACCENT_HI`. Press → ×0.9 brightness.
|
||||
- *Recording:* `T_DANGER`, breathing alpha, square "stop" glyph, label "Stop", and a **live waveform** drawn across the pill or in the strip below.
|
||||
- Keep it the visual anchor; everything else is quieter.
|
||||
|
||||
**Ghost actions (Copy / Paste / Clear).** No resting fill, no border — just a Medium-weight label in `T_DIM`. On hover, a `T_CARD_HI` rounded fill fades in and text lifts to `T_TEXT`; on press, `T_CARD_LO`. Because there's no resting border, there are no hairlines; separation is purely spacing. (Add small 16px line icons before labels for a more finished feel.)
|
||||
|
||||
**Pin.** An icon toggle (pin glyph), `T_ACCENT` when active, `T_FAINT` when not. No label needed.
|
||||
|
||||
**Selects (mic / model).** Quiet rows: small dim label on top ("Microphone"), value below in `T_TEXT`, a small chevron at the right; hover = `T_CARD_HI` fill. **Consider relocating both behind a small gear/settings affordance** — a dictation utility doesn't need model internals on the main face. If you keep them visible, give them the same fill-on-hover, no-border treatment.
|
||||
|
||||
**Status + progress strip (unified).** One horizontal zone under the hero that changes by state:
|
||||
- *Idle:* `"Ready · 2 threads"` in `T_DIM`.
|
||||
- *Recording:* live waveform + `mm:ss` timer.
|
||||
- *Transcribing:* the progress bar (Part 2) with smooth % and a **counting-down** ETA.
|
||||
|
||||
**Level / waveform.** Replace the 14-segment VU (reads as "old") with either a smooth antialiased waveform (ring buffer of recent RMS samples drawn as a filled path) or a single breathing level pill. Color `T_GOOD`, riding on `T_CARD_LO`.
|
||||
|
||||
**Empty state.** Centered mic glyph + "Your transcription will appear here" in `T_DIM`, drawn *inside* the card (not as a separate STATIC) so it shares the surface.
|
||||
|
||||
## 1.6 Rendering scaffold + hit-testing (code)
|
||||
|
||||
The whole UI becomes a small list of widgets plus one paint routine and one interaction handler. Skeleton (GDI+, Tier 1):
|
||||
|
||||
```cpp
|
||||
enum class WK { RecordHero, Pin, Copy, Paste, Clear, SelAudio, SelModel, Transcript };
|
||||
|
||||
struct Widget {
|
||||
WK kind;
|
||||
RectF r; // logical rect, filled by Layout()
|
||||
bool hover = false;
|
||||
bool pressed = false;
|
||||
float anim = 0.0f; // 0..1 eased hover/press amount
|
||||
};
|
||||
|
||||
static Widget g_w[ (int)WK::Transcript + 1 ];
|
||||
static int g_hot = -1; // index under cursor
|
||||
static int g_active = -1; // index pressed
|
||||
|
||||
// --- one animation clock ---
|
||||
static DWORD g_lastFrame = 0;
|
||||
static bool AnyAnimating(); // true if any widget anim is mid-transition, or recording, or busy
|
||||
|
||||
// Advance eased states; call from the render timer.
|
||||
void StepAnimations(float dt) {
|
||||
for (auto& w : g_w) {
|
||||
float target = (g_active == (&w - g_w) ) ? 1.0f : (w.hover ? 0.6f : 0.0f);
|
||||
// ease toward target
|
||||
w.anim += (target - w.anim) * std::min(1.0f, dt * 12.0f);
|
||||
}
|
||||
}
|
||||
|
||||
// --- layout: compute rects from client size & DPI scale ---
|
||||
void Layout(int W, int H, float s /*dpi scale*/);
|
||||
|
||||
// --- paint: ONE surface ---
|
||||
void Paint(HWND hwnd) {
|
||||
PAINTSTRUCT ps; HDC hdc = BeginPaint(hwnd, &ps);
|
||||
RECT rc; GetClientRect(hwnd, &rc);
|
||||
int W = rc.right, H = rc.bottom;
|
||||
|
||||
HDC mem = CreateCompatibleDC(hdc);
|
||||
HBITMAP bmp = CreateCompatibleBitmap(hdc, W, H);
|
||||
HBITMAP old = (HBITMAP)SelectObject(mem, bmp);
|
||||
{
|
||||
Graphics g(mem);
|
||||
g.SetSmoothingMode(SmoothingModeAntiAlias);
|
||||
g.SetTextRenderingHint(TextRenderingHintClearTypeGridFit);
|
||||
|
||||
SolidBrush bg(T_BG); g.FillRectangle(&bg, 0, 0, W, H);
|
||||
|
||||
DrawCardWithShadow(g, g_cardRect, 16); // optional soft shadow + T_CARD fill + T_TOPLIGHT edge
|
||||
|
||||
for (auto& w : g_w) {
|
||||
switch (w.kind) {
|
||||
case WK::RecordHero: DrawHero(g, w); break;
|
||||
case WK::Copy: DrawGhost(g, w, L"Copy"); break;
|
||||
case WK::Paste: DrawGhost(g, w, L"Paste"); break;
|
||||
case WK::Clear: DrawGhost(g, w, L"Clear"); break;
|
||||
case WK::Pin: DrawPin(g, w); break;
|
||||
case WK::SelAudio: DrawSelect(g, w, g_audioVal); break;
|
||||
case WK::SelModel: DrawSelect(g, w, g_modelVal); break;
|
||||
case WK::Transcript: /* the EDIT child paints itself; we just leave its inset */ break;
|
||||
}
|
||||
}
|
||||
DrawStatusStrip(g, g_stripRect); // idle / recording waveform / progress
|
||||
}
|
||||
BitBlt(hdc, 0, 0, W, H, mem, 0, 0, SRCCOPY);
|
||||
SelectObject(mem, old); DeleteObject(bmp); DeleteDC(mem);
|
||||
EndPaint(hwnd, &ps);
|
||||
}
|
||||
|
||||
// --- interaction: hit-test in the parent ---
|
||||
int HitTest(POINT p) {
|
||||
for (int i = 0; i < (int)std::size(g_w); ++i)
|
||||
if (g_w[i].kind != WK::Transcript && g_w[i].r.Contains((REAL)p.x, (REAL)p.y)) return i;
|
||||
return -1;
|
||||
}
|
||||
|
||||
LRESULT CALLBACK WndProc(HWND h, UINT m, WPARAM w, LPARAM l) {
|
||||
switch (m) {
|
||||
case WM_MOUSEMOVE: {
|
||||
POINT p{ GET_X_LPARAM(l), GET_Y_LPARAM(l) };
|
||||
int hot = HitTest(p);
|
||||
if (hot != g_hot) {
|
||||
if (g_hot >= 0) g_w[g_hot].hover = false;
|
||||
g_hot = hot;
|
||||
if (g_hot >= 0) g_w[g_hot].hover = true;
|
||||
TRACKMOUSEEVENT t{ sizeof(t), TME_LEAVE, h, 0 }; TrackMouseEvent(&t);
|
||||
EnsureAnimating(h);
|
||||
}
|
||||
return 0;
|
||||
}
|
||||
case WM_MOUSELEAVE:
|
||||
if (g_hot >= 0) { g_w[g_hot].hover = false; g_hot = -1; EnsureAnimating(h); }
|
||||
return 0;
|
||||
case WM_LBUTTONDOWN:
|
||||
g_active = g_hot;
|
||||
if (g_active >= 0) { g_w[g_active].pressed = true; SetCapture(h); EnsureAnimating(h); }
|
||||
return 0;
|
||||
case WM_LBUTTONUP: {
|
||||
ReleaseCapture();
|
||||
POINT p{ GET_X_LPARAM(l), GET_Y_LPARAM(l) };
|
||||
if (g_active >= 0 && HitTest(p) == g_active) OnClick(h, g_w[g_active].kind);
|
||||
if (g_active >= 0) g_w[g_active].pressed = false;
|
||||
g_active = -1; EnsureAnimating(h);
|
||||
return 0;
|
||||
}
|
||||
case WM_ERASEBKGND: return 1; // we paint everything
|
||||
case WM_PAINT: Paint(h); return 0;
|
||||
case WM_SIZE: Layout(LOWORD(l), HIWORD(l), g_dpiScale); InvalidateRect(h, nullptr, FALSE); return 0;
|
||||
// ... WM_TIMER drives StepAnimations + InvalidateRect while AnyAnimating()
|
||||
}
|
||||
return DefWindowProc(h, m, w, l);
|
||||
}
|
||||
```
|
||||
|
||||
Notes:
|
||||
- `DrawGhost` simply lerps its fill alpha by `w.anim` between transparent → `T_CARD_HI`, and text color between `T_DIM` → `T_TEXT`. No `StrokeRound`. That's the whole trick.
|
||||
- `EnsureAnimating(h)` starts the 16ms timer if it isn't running; the timer stops itself when `AnyAnimating()` returns false to spare the CPU.
|
||||
- Keyboard focus (for accessibility / Tab) can be added later by tracking a `g_focus` index and painting a soft focus ring on the focused widget only — still no native chrome.
|
||||
|
||||
## 1.7 The transcript field & DPI
|
||||
|
||||
**Transcript = the one real child window.** Keep `EDIT` (multiline, read-only) for free selection/caret/scroll/IME, but strip its chrome:
|
||||
|
||||
1. **No border:** create without `WS_BORDER`/`WS_EX_CLIENTEDGE` (already the case). To suppress the *themed* edit border entirely, either `SetWindowTheme(hEdit, L"", L"")` (kills the theme, gives a classic flat look) or subclass and handle `WM_NCPAINT` to no-op. Prefer the dark-mode route below so the scrollbar also matches.
|
||||
2. **Dark background:** you already return `g_brSurface` from `WM_CTLCOLOREDIT`; set it to `T_CARD`/`T_CARD_LO` so the field is invisible against the card.
|
||||
3. **Dark (or custom) scrollbar — this removes the pale bar in your screenshot:**
|
||||
- Easiest: enable app dark mode then theme the control:
|
||||
```cpp
|
||||
// once, after the process starts (uxtheme, undocumented but widely used):
|
||||
// AllowDarkModeForApp(true); SetPreferredAppMode(AllowDark);
|
||||
SetWindowTheme(hEdit, L"DarkMode_Explorer", nullptr); // dark scrollbar
|
||||
```
|
||||
- Most control: hide the native scrollbar (`ShowScrollBar(hEdit, SB_VERT, FALSE)` or `WM_NCCALCSIZE`) and **paint a slim custom scrollbar on the parent surface**, driven by `EM_GETFIRSTVISIBLELINE` / line count. Best looking, more work.
|
||||
4. **Inset it inside the card** by ~14–16px so the card's rounded surface is the visible frame and the EDIT contributes no edges of its own.
|
||||
|
||||
**DPI awareness (do this — it's part of "looks good").** Today metrics are fixed pixels; on a HiDPI panel they blur/misalign.
|
||||
|
||||
- Declare **Per-Monitor-V2** via the app manifest (preferred) or `SetProcessDpiAwarenessContext(DPI_AWARENESS_CONTEXT_PER_MONITOR_AWARE_V2)` at startup.
|
||||
- Compute `g_dpiScale = GetDpiForWindow(hwnd) / 96.0f`; multiply every metric (padding, radii, font sizes, widget sizes) by it.
|
||||
- Recreate fonts and re-`Layout()` on `WM_DPICHANGED`, and use the suggested rect it passes for repositioning.
|
||||
|
||||
## 1.8 Migration order from today's main.cpp
|
||||
|
||||
You can do this incrementally without a rewrite:
|
||||
|
||||
1. **Stop creating chrome child windows.** Delete the `CreateWindow(L"BUTTON", …)` calls for Record/Pin/Copy/Paste/Clear and the two selects, and the two `STATIC`s. Keep only the `EDIT`.
|
||||
2. **Add the `Widget` array + `Layout()`** computing the same rectangles your `LayoutControls` used (reuse the math; just store `RectF`s instead of `MoveWindow`-ing HWNDs).
|
||||
3. **Move your existing draw functions** (`DrawRecordButton`→`DrawHero`, `DrawFlatButton`→`DrawGhost`, `DrawSelect`) to take a `Widget&` and draw into the shared `Graphics&` — and **delete every `StrokeRound(..., T_BORDER/C_BORDER, …)`**. Replace the panel border with the card + optional shadow + toplight.
|
||||
4. **Route interaction** through `WndProc` hit-testing (Section 1.6). `OnClick(kind)` calls your existing handlers (toggle record, copy, paste, clear, open the popup for selects).
|
||||
5. **Add the animation clock**; convert hover from per-HWND `GWLP_USERDATA` to `w.anim`.
|
||||
6. **Theme the EDIT + scrollbar**, inset it, and add DPI scaling.
|
||||
7. The popup list (`PopupProc`) can stay as-is — it's already a single custom-painted surface and looks consistent.
|
||||
|
||||
Result: one surface, zero seams, full control. The "thin lines" cannot come back because nothing draws them and there are no child boundaries to leak them.
|
||||
|
||||
---
|
||||
|
||||
# Part 2 — The progress system
|
||||
|
||||
## 2.1 Why it's broken today
|
||||
|
||||
Three separate problems, all visible in your 1:27 example.
|
||||
|
||||
**(a) The ETA counts *up*.** In `UpdateStatus`:
|
||||
|
||||
```cpp
|
||||
float elapsed = (GetTickCount() - g_busyStart) / 1000.0f;
|
||||
float est = elapsed * 100.0f / (float)p; // total, derived from stale p
|
||||
float remain = est - elapsed; // = elapsed * (100 - p) / p
|
||||
```
|
||||
|
||||
`remain` is recomputed every 50ms, but `p` (whisper's progress) only changes at chunk boundaries. With `p` held constant and `elapsed` rising, `remain = elapsed·(100−p)/p` **increases over time** — the ETA climbs until the next `p` update, then snaps down when `p` jumps. That's precisely "counts up, then jumps to 20s, then counts up again."
|
||||
|
||||
**(b) The percentage jumps in big steps.** whisper.cpp calls its progress callback at most **once per 30-second audio chunk**. 1:27 = 87s ≈ **three chunks**, so `p` arrives roughly as `0 → 33 → 67 → 100`. The 34% and 72% you saw are those chunk boundaries (off slightly due to seek rounding). The bar can't be smooth if its only input updates 3 times.
|
||||
|
||||
**(c) Dead air at the start.** For 87s of audio the first callback only fires after the *first* 30s chunk finishes decoding — several seconds on a 2-core CPU — so nothing moves at first (you read it as "model loading"). The model is actually already preloaded; it's first-chunk latency with no fallback signal.
|
||||
|
||||
**Conclusion:** whisper's callback is a *coarse, occasional measurement*, not a progress source. We need our own continuous prediction, corrected by that measurement.
|
||||
|
||||
## 2.2 The plan: predict, then correct
|
||||
|
||||
Exactly your idea, formalized:
|
||||
|
||||
1. **Predict** total processing time the instant recording stops, from a **history of how long this machine took** for clips of various lengths (per model). This drives a smooth bar from frame 1 — even for sub-30s clips that get *zero* whisper updates.
|
||||
2. **Correct** that prediction as whisper reports progress: each callback implies a *measured* total time; we fuse it into our estimate with exponential smoothing so accuracy improves **without jumps**.
|
||||
3. **Display** a strictly **counting-down** remaining time and a **smoothly rising** percent derived from the same model, ease to **95%**, and **snap to 100%** when the real result arrives.
|
||||
4. **Learn:** on completion, record `(audio_seconds, actual_processing_seconds)` and persist it, so the next prediction is better.
|
||||
|
||||
## 2.3 Persistent per-model timing history
|
||||
|
||||
Processing time vs audio length is, to first order, **linear**: `proc ≈ a + b·audio`, where `b` is roughly the inverse real-time factor and `a` is fixed overhead. We fit `a, b` per model (tiny.en and base.en behave very differently) with an **online least-squares** accumulator, with a gentle decay so the model adapts to thermal throttling / machine load.
|
||||
|
||||
- **Key by model filename** (e.g. `ggml-tiny.en.bin`), since speed is model-dependent.
|
||||
- **Cold start:** before we have ≥2 samples, use baked-in defaults (rough seeds for a 2-core i5-7th-gen; they self-correct after a run or two):
|
||||
- tiny.en: `a ≈ 0.3s`, `b ≈ 0.45` (≈2.2× real-time)
|
||||
- base.en: `a ≈ 0.5s`, `b ≈ 1.1` (≈0.9× real-time)
|
||||
- (These are only seeds; the regression takes over quickly.)
|
||||
- **Persist** alongside the existing `win-dictation.ini` using the same `WritePrivateProfileString` style you already use in `settings.h`, one section per model holding the five accumulators.
|
||||
|
||||
## 2.4 The live estimator (smooth countdown + fusion)
|
||||
|
||||
State: `T_hat` (current best total-time estimate), `disp_rem` (displayed remaining, monotonic), `t` (seconds since start).
|
||||
|
||||
- **begin(T_pred):** `T_hat = disp_rem = max(0.4, T_pred)`, `t = 0`.
|
||||
- **on_whisper(t_now, p):** ignore `p < 5` (noisy). Else measured total `T_meas = 100·t_now / p`; fuse: `T_hat = (1−α)·T_hat + α·T_meas` with `α ≈ 0.5`. This is where whisper "adjusts our countdown" — it moves the estimate, not the displayed number directly, so there's never a visible jump.
|
||||
- **tick(dt):** the smoothing rules that make it feel solid:
|
||||
1. Always count down in real time: `disp_rem -= dt`.
|
||||
2. Pull toward the model's `raw_rem = max(0, T_hat − t)`, but **only ever downward**, and **rate-limited**:
|
||||
- `err = raw_rem − disp_rem`
|
||||
- if `err < 0` (we're behind → need to speed up): `disp_rem += max(err, −maxCatchUp·dt)` (bounded extra shrink, no snap)
|
||||
- if `err ≥ 0` (we have more headroom than shown): **do nothing** — never push remaining up. The bar simply keeps easing and parks near 95% if we under-predicted.
|
||||
3. Clamp `disp_rem ≥ 0`.
|
||||
4. Derive fraction from the same numbers: `frac = t / (t + disp_rem)`, clamp to **0.95**. Because `t` only rises and `disp_rem` only falls, `frac` only rises — smooth, monotonic, no jumps.
|
||||
- **on_result:** snap `frac → 1.0`; record `(audio_seconds, t)` into the timing model and persist.
|
||||
|
||||
This guarantees: **ETA only counts down** (bug fixed), **% only rises smoothly** (no 34→72 jumps), whisper's coarse measurements **gently re-aim** the countdown, and there's **motion from frame 1** (no dead start). On a sub-30s clip with no whisper updates, it runs purely on the learned prediction — exactly what you asked for.
|
||||
|
||||
## 2.5 `timing.h` — full code
|
||||
|
||||
```cpp
|
||||
#pragma once
|
||||
#include <windows.h>
|
||||
#include <string>
|
||||
#include <cmath>
|
||||
#include <algorithm>
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Online linear model: proc_sec ~= a + b * audio_sec, fitted per whisper model.
|
||||
// Decayed least squares so it adapts to thermal / load drift over time.
|
||||
// ---------------------------------------------------------------------------
|
||||
struct TimingModel {
|
||||
double n=0, sx=0, sy=0, sxx=0, sxy=0; // decayed accumulators
|
||||
double a=0, b=0; // fitted intercept / slope
|
||||
bool fitted=false;
|
||||
double def_a=0.4, def_b=0.6; // cold-start seeds (set per model)
|
||||
|
||||
void recompute() {
|
||||
if (n >= 2.0) {
|
||||
double denom = n*sxx - sx*sx;
|
||||
if (std::fabs(denom) > 1e-9) {
|
||||
double bb = (n*sxy - sx*sy) / denom;
|
||||
double aa = (sy - bb*sx) / n;
|
||||
if (bb < 0.02) bb = def_b; // guard against degenerate fits
|
||||
if (aa < 0.0) aa = 0.0;
|
||||
a=aa; b=bb; fitted=true; return;
|
||||
}
|
||||
}
|
||||
a=def_a; b=def_b; fitted=false;
|
||||
}
|
||||
|
||||
double predict(double audio_sec) const {
|
||||
double t = (fitted ? a : def_a) + (fitted ? b : def_b) * audio_sec;
|
||||
return std::max(0.4, t);
|
||||
}
|
||||
|
||||
void add_sample(double audio_sec, double proc_sec) {
|
||||
const double decay = 0.97; // ~30-sample memory
|
||||
n*=decay; sx*=decay; sy*=decay; sxx*=decay; sxy*=decay;
|
||||
n+=1; sx+=audio_sec; sy+=proc_sec;
|
||||
sxx+=audio_sec*audio_sec; sxy+=audio_sec*proc_sec;
|
||||
recompute();
|
||||
}
|
||||
};
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Live estimator: smooth, monotonic countdown fused with whisper's progress.
|
||||
// ---------------------------------------------------------------------------
|
||||
struct ProgressEstimator {
|
||||
double T_hat=1.0, disp_rem=1.0, t=0.0;
|
||||
bool done=false;
|
||||
|
||||
void begin(double T_pred) {
|
||||
T_hat = std::max(0.4, T_pred);
|
||||
disp_rem = T_hat; t = 0.0; done=false;
|
||||
}
|
||||
|
||||
void on_whisper(double t_now, int p) { // p in (0,100]
|
||||
if (done || p < 5) return;
|
||||
double T_meas = 100.0 * t_now / (double)p;
|
||||
const double alpha = 0.5; // how much we trust the measurement
|
||||
T_hat = (1.0-alpha)*T_hat + alpha*T_meas;
|
||||
if (T_hat < t_now) T_hat = t_now; // never imply we're already done
|
||||
}
|
||||
|
||||
// dt seconds since last tick. Outputs eased fraction [0,1] and remaining secs.
|
||||
void tick(double dt, float& out_frac, float& out_remaining) {
|
||||
if (done) { out_frac=1.0f; out_remaining=0.0f; return; }
|
||||
t += dt;
|
||||
disp_rem -= dt; // (1) real-time countdown
|
||||
double raw_rem = std::max(0.0, T_hat - t);
|
||||
const double maxCatchUp = 2.5; // cap speed-up (×realtime)
|
||||
double err = raw_rem - disp_rem;
|
||||
if (err < 0) disp_rem += std::max(err, -maxCatchUp*dt); // (2) shrink only
|
||||
if (disp_rem < 0) disp_rem = 0; // (3)
|
||||
double frac = (t + disp_rem > 1e-6) ? t/(t+disp_rem) : 0.0;
|
||||
if (frac > 0.95) frac = 0.95; // (4) hold until result
|
||||
out_frac = (float)frac;
|
||||
out_remaining = (float)disp_rem;
|
||||
}
|
||||
|
||||
void finish(float& out_frac, float& out_remaining) {
|
||||
done=true; out_frac=1.0f; out_remaining=0.0f;
|
||||
}
|
||||
};
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Persistence (same ini style as settings.h). Section = model base filename.
|
||||
// ---------------------------------------------------------------------------
|
||||
inline std::wstring TimingIniPath() {
|
||||
wchar_t buf[MAX_PATH]; GetModuleFileNameW(nullptr, buf, MAX_PATH);
|
||||
std::wstring p(buf); p = p.substr(0, p.find_last_of(L"\\/"));
|
||||
return p + L"\\win-dictation.ini";
|
||||
}
|
||||
inline std::wstring SectionFor(const std::string& modelPath) {
|
||||
std::string base = modelPath.substr(modelPath.find_last_of("\\/")+1);
|
||||
return L"timing-" + std::wstring(base.begin(), base.end());
|
||||
}
|
||||
inline void PutD(const std::wstring& sec, const wchar_t* k, double v) {
|
||||
wchar_t b[64]; swprintf_s(b, L"%.6f", v);
|
||||
WritePrivateProfileStringW(sec.c_str(), k, b, TimingIniPath().c_str());
|
||||
}
|
||||
inline double GetD(const std::wstring& sec, const wchar_t* k, double d) {
|
||||
wchar_t b[64]; swprintf_s(b, L"%.6f", d);
|
||||
wchar_t out[64];
|
||||
GetPrivateProfileStringW(sec.c_str(), k, b, out, 64, TimingIniPath().c_str());
|
||||
return wcstod(out, nullptr);
|
||||
}
|
||||
inline void LoadTiming(TimingModel& m, const std::string& modelPath) {
|
||||
auto s = SectionFor(modelPath);
|
||||
m.n=GetD(s,L"n",0); m.sx=GetD(s,L"sx",0); m.sy=GetD(s,L"sy",0);
|
||||
m.sxx=GetD(s,L"sxx",0); m.sxy=GetD(s,L"sxy",0);
|
||||
m.recompute();
|
||||
}
|
||||
inline void SaveTiming(const TimingModel& m, const std::string& modelPath) {
|
||||
auto s = SectionFor(modelPath);
|
||||
PutD(s,L"n",m.n); PutD(s,L"sx",m.sx); PutD(s,L"sy",m.sy);
|
||||
PutD(s,L"sxx",m.sxx); PutD(s,L"sxy",m.sxy);
|
||||
}
|
||||
|
||||
// Set per-model cold-start seeds when (re)loading a model.
|
||||
inline void SeedDefaults(TimingModel& m, const std::string& modelPath) {
|
||||
std::string p = modelPath;
|
||||
auto has = [&](const char* s){ return p.find(s)!=std::string::npos; };
|
||||
if (has("tiny")) { m.def_a=0.3; m.def_b=0.45; }
|
||||
else if (has("base")) { m.def_a=0.5; m.def_b=1.10; }
|
||||
else if (has("small")){ m.def_a=0.8; m.def_b=3.00; }
|
||||
else { m.def_a=0.5; m.def_b=1.00; }
|
||||
m.recompute();
|
||||
}
|
||||
```
|
||||
|
||||
## 2.6 Wiring into main.cpp
|
||||
|
||||
Add globals and capture the audio length **before** `stop_and_transcribe()` swaps the buffer away:
|
||||
|
||||
```cpp
|
||||
TimingModel g_timing;
|
||||
ProgressEstimator g_est;
|
||||
double g_lastAudioLen = 0.0; // seconds of the clip being transcribed
|
||||
DWORD g_lastTick = 0;
|
||||
```
|
||||
|
||||
**At model load / model switch** (where you set `g_config.model_path`), seed + load history:
|
||||
|
||||
```cpp
|
||||
SeedDefaults(g_timing, g_config.model_path);
|
||||
LoadTiming(g_timing, g_config.model_path);
|
||||
```
|
||||
|
||||
**On STOP → transcribe** (the `HK_TOGGLE` stop branch and the max-length branch):
|
||||
|
||||
```cpp
|
||||
g_lastAudioLen = g_tx.recorded_seconds(); // BEFORE stop swaps the buffer
|
||||
g_busyStart = GetTickCount();
|
||||
g_lastTick = g_busyStart;
|
||||
g_est.begin(g_timing.predict(g_lastAudioLen)); // bar moves from frame 1
|
||||
g_progress = 0;
|
||||
g_cancelRequested = false;
|
||||
g_tx.stop_and_transcribe();
|
||||
```
|
||||
|
||||
**whisper progress** (`WM_APP_PROGRESS`) becomes a *correction*, not the display source:
|
||||
|
||||
```cpp
|
||||
case WM_APP_PROGRESS: {
|
||||
double t_now = (GetTickCount() - g_busyStart) / 1000.0;
|
||||
g_est.on_whisper(t_now, (int)wParam);
|
||||
InvalidateRect(hWnd, &g_vuRect, FALSE);
|
||||
return 0;
|
||||
}
|
||||
```
|
||||
|
||||
**The UI timer** (`WM_TIMER`, while `g_tx.is_busy()`) advances the estimator and paints:
|
||||
|
||||
```cpp
|
||||
DWORD now = GetTickCount();
|
||||
float dt = (now - g_lastTick) / 1000.0f; g_lastTick = now;
|
||||
float frac, remain;
|
||||
g_est.tick(dt, frac, remain);
|
||||
g_progressFrac = frac; // float 0..1 used by DrawProgress
|
||||
g_progressRemain = remain; // seconds, for the ETA label
|
||||
InvalidateRect(hWnd, &g_vuRect, FALSE);
|
||||
```
|
||||
|
||||
**Status text** (replaces the counts-up math entirely):
|
||||
|
||||
```cpp
|
||||
int mm = (int)g_lastAudioLen/60, ss=(int)g_lastAudioLen%60;
|
||||
int pct = (int)(g_progressFrac*100.0f + 0.5f);
|
||||
int rem = (int)(g_progressRemain + 0.5f);
|
||||
swprintf_s(buf, L"Transcribing %d:%02d · %d%% · %ds left", mm, ss, pct, rem);
|
||||
```
|
||||
|
||||
**On result** (`WM_APP_RESULT`): snap, learn, persist:
|
||||
|
||||
```cpp
|
||||
float frac, remain; g_est.finish(frac, remain);
|
||||
g_progressFrac = 1.0f; g_progressRemain = 0.0f;
|
||||
double actual = (GetTickCount() - g_busyStart) / 1000.0;
|
||||
if (g_lastAudioLen > 0.5 && actual > 0.2 && !g_cancelRequested) {
|
||||
g_timing.add_sample(g_lastAudioLen, actual);
|
||||
SaveTiming(g_timing, g_config.model_path);
|
||||
}
|
||||
```
|
||||
|
||||
**`DrawProgress`** already takes a fraction — just feed `g_progressFrac` instead of `g_progress/100.0f`, and optionally add a subtle animated shimmer on the fill for life.
|
||||
|
||||
## 2.7 Tuning & edge cases
|
||||
|
||||
- **`alpha` (whisper trust)** 0.5 is a good start. Lower (0.3) = smoother but slower to correct; higher (0.7) = snappier, slightly jumpier.
|
||||
- **`maxCatchUp`** (2.5× real-time) caps how fast the countdown may accelerate when we over-predicted, so a correction never looks like a snap. Raise for faster catch-up, lower for calmer motion.
|
||||
- **Under-prediction** (transcription takes longer than estimated): `frac` parks at 95% and the ETA sits at a small floor until the result lands — which is the honest, expected behavior.
|
||||
- **Decay `0.97`** ≈ last ~30 runs dominate. Increase toward 0.99 for steadier long-term averages, decrease for faster adaptation to a throttling machine.
|
||||
- **Cancel / no-speech:** call `g_est.finish(...)`, reset `g_progressFrac=0`, and **don't** record a sample.
|
||||
- **Model switch mid-history:** because history is keyed per model, switching tiny.en↔base.en uses the right curve automatically.
|
||||
- **Sanity clamp:** keep `predict()`'s floor (0.4s) so ultra-short clips still show a brief, graceful sweep rather than instant 100%.
|
||||
- **Optional richer model:** if you ever want better fits on very short vs long clips, swap the linear `a+b·x` for a two-segment fit (sub-30s vs ≥30s) — the accumulators and API stay the same; just keep two `TimingModel`s.
|
||||
|
||||
---
|
||||
|
||||
# Part 3 — Cleanup checklist
|
||||
|
||||
Smaller items that make the project cleaner and the app feel finished:
|
||||
|
||||
- [ ] **Kill the pale scrollbar** (dark-mode theme or custom slim scrollbar) — Section 1.7.
|
||||
- [ ] **Delete all owner-draw chrome child windows**; keep only the transcript `EDIT` — Section 1.8.
|
||||
- [ ] **Remove every `StrokeRound(..., C_BORDER, …)`**; rely on fills + elevation.
|
||||
- [ ] **One animation clock** that idles when nothing moves (protect the 2-core CPU).
|
||||
- [ ] **DPI Per-Monitor-V2** + scaled metrics + font reload on `WM_DPICHANGED`.
|
||||
- [ ] **Reconcile the docs with reality.** `README.md`, `src/README.md`, and `src/CHANGES.md` still describe the *old* streaming architecture — "ring buffer", "24 threads", "VAD", "step_ms/length_ms", "<1s real-time GPU". The app is now **push-to-talk batch, CPU, physical-core threads, `whisper_full` once on stop**. Update or archive those docs so future-you isn't misled. (The `CUDA-SETUP.md` / `QUICK-REBUILD-GPU.md` RTX-3090 guides don't apply to the target Dell i5 either.)
|
||||
- [ ] **Status copy:** "Ready · N threads" is good; make the idle/recording/transcribing strings come from one place.
|
||||
- [ ] **Remove dead members** once streaming is gone (any leftover `step_ms`/`length_ms`/VAD config that no longer feeds `whisper_full`).
|
||||
|
||||
---
|
||||
|
||||
# Part 4 — Suggested build order
|
||||
|
||||
Do them in this sequence so each step is verifiable on its own:
|
||||
|
||||
1. **Progress system first** (Part 2). It's self-contained, low-risk, and immediately fixes the most visible "is it even working?" problem. You'll see a smooth countdown the same day.
|
||||
2. **DPI + tokens** (1.4 / 1.7). Small, mechanical, and everything after looks better for it.
|
||||
3. **Single-surface conversion** (1.6 / 1.8): convert one widget at a time — start with the ghost buttons (highest hairline payoff), then the hero, then the selects, then retire the STATICs.
|
||||
4. **Transcript chrome + scrollbar** (1.7).
|
||||
5. **Motion polish** (1.4): hover cross-fades, recording breathing, waveform, progress shimmer.
|
||||
6. **Docs reconciliation** (Part 3).
|
||||
7. *(Optional later)* **Direct2D/DirectWrite** (1.3) if you want the premium ceiling — the data model and Part 2 carry over unchanged.
|
||||
|
||||
---
|
||||
|
||||
*Build target reminder: native Win32 C++, MSVC Release, CPU-only on a 2-core / 4-thread i5-7th-gen. Keep the idle CPU near zero — animate only when something is actually moving.*
|
||||
Reference in New Issue
Block a user