v2 rebuild: GDI+ single-surface UI, self-calibrating progress, compact push-to-talk, GGML+Whisper integration
This commit is contained in:
Win Dictation Dev
+26
-295
@@ -1,308 +1,39 @@
|
||||
# Whisper Dictation v2.0 - Changelog
|
||||
# Win Dictation — Changelog
|
||||
|
||||
## Overview
|
||||
Complete rewrite of win-dictation with focus on performance, reliability, and user experience.
|
||||
## v3.0 — Architecture & UI Rebuild (current)
|
||||
|
||||
## Major Changes
|
||||
### Single-surface UI
|
||||
|
||||
### 1. Audio Capture System (Zero Loss)
|
||||
- Replaced 9 separate child windows with one immediate-mode painted surface
|
||||
- Eliminated inter-window hairline seams at the root
|
||||
- Widget data model with hover/press animation blending
|
||||
- Per-monitor DPI scaling (V2)
|
||||
- Dark mode design tokens
|
||||
- Dark scrollbar on the transcript edit control
|
||||
|
||||
**Problem:** Audio chunks were being dropped between recording and processing.
|
||||
### Progress system
|
||||
|
||||
**Solution:** Implemented lock-free ring buffer system
|
||||
- 30-second circular buffer (480K samples)
|
||||
- Atomic read/write positions
|
||||
- No mutex in audio callback
|
||||
- Handles burst processing gracefully
|
||||
- Self-calibrating linear estimator: `proc = a + b·audio`, fitted per model with decayed online least-squares
|
||||
- Persisted per-model timing history in `win-dictation.ini`
|
||||
- Strictly monotonic countdown (never counts up)
|
||||
- Smooth percent that eases, never snaps (except final 100%)
|
||||
- Motion from frame 1 (predicts before whisper reports)
|
||||
|
||||
**Files Changed:**
|
||||
- `transcriber.h`: Added ring buffer members
|
||||
- `transcriber.cpp`: Rewrote audio_callback() and worker_loop()
|
||||
### Push-to-talk batch mode
|
||||
|
||||
### 2. Multi-Core CPU Support
|
||||
- Removed streaming/VAD/ring-buffer architecture
|
||||
- Record `→` stop `→` single `whisper_full` call
|
||||
- 500ms silence auto-end timer
|
||||
- Physical-core threading (matches the i5 target)
|
||||
|
||||
**Problem:** Only using 1-2 CPU cores despite having 24 available.
|
||||
### Files
|
||||
|
||||
**Solution:** Proper thread configuration
|
||||
- Uses `std::thread::hardware_concurrency()` (24 threads)
|
||||
- OpenMP support enabled in build
|
||||
- Optimized work distribution
|
||||
|
||||
**Configuration:**
|
||||
```cpp
|
||||
WhisperConfig::n_threads = std::thread::hardware_concurrency(); // 24
|
||||
```
|
||||
|
||||
### 3. GPU Acceleration
|
||||
|
||||
**Problem:** No GPU utilization despite CUDA installation.
|
||||
|
||||
**Solution:** GPU detection and backend selection
|
||||
- Auto-detects CUDA/Vulkan/Metal
|
||||
- Graceful fallback to CPU
|
||||
- Version compatibility checking
|
||||
- Status display in UI
|
||||
|
||||
**Build Script:**
|
||||
- Detects GPU capabilities
|
||||
- Checks CUDA version compatibility (11.7 vs 12.4 requirement)
|
||||
- Falls back to optimized CPU build
|
||||
|
||||
### 4. Modern User Interface
|
||||
|
||||
**Problem:** Slow, glitchy interface with poor visual feedback.
|
||||
|
||||
**Solution:** Complete UI overhaul
|
||||
- Dark theme with modern colors
|
||||
- 30 FPS update timer (was 50ms/20 FPS)
|
||||
- Custom button drawing
|
||||
- Real-time status indicators
|
||||
- Smooth animations
|
||||
|
||||
**UI Features:**
|
||||
- VU meter (real-time audio level)
|
||||
- Buffer indicator (queue status)
|
||||
- GPU/CPU status
|
||||
- Thread count display
|
||||
- Responsive layout
|
||||
|
||||
**Colors:**
|
||||
```cpp
|
||||
#define COLOR_BG RGB(32, 33, 36)
|
||||
#define COLOR_PRIMARY RGB(138, 180, 248)
|
||||
#define COLOR_SUCCESS RGB(129, 201, 149)
|
||||
```
|
||||
|
||||
### 5. Processing Optimizations
|
||||
|
||||
**Changes:**
|
||||
- Reduced step_ms: 3000ms → 1500ms (faster response)
|
||||
- Reduced length_ms: 10000ms → 8000ms (better streaming)
|
||||
- Added VAD filtering (skip silence)
|
||||
- Improved context handling
|
||||
- Better memory management
|
||||
|
||||
### 6. Build System
|
||||
|
||||
**New Features:**
|
||||
- Automated GPU detection
|
||||
- Version compatibility checking
|
||||
- One-command build and deploy
|
||||
- Automatic model download
|
||||
- DLL deployment
|
||||
|
||||
**Script:** `build.ps1`
|
||||
```powershell
|
||||
# Detects:
|
||||
- NVIDIA GPU + CUDA version
|
||||
- AMD GPU + ROCm
|
||||
- Vulkan SDK
|
||||
- CPU capabilities (AVX2/FMA)
|
||||
```
|
||||
|
||||
## File-by-File Changes
|
||||
|
||||
### transcriber.h
|
||||
```diff
|
||||
+ Ring buffer implementation (RING_BUFFER_SIZE = 480K)
|
||||
+ Atomic position tracking
|
||||
+ get_buffer_fullness() method
|
||||
+ is_using_gpu() const correctness
|
||||
+ Processing buffer for context
|
||||
+ GPU active flag
|
||||
- Simple deque queue
|
||||
- Blocking mutex in callback
|
||||
```
|
||||
|
||||
### transcriber.cpp
|
||||
```diff
|
||||
+ Lock-free ring buffer audio_callback()
|
||||
+ Atomic read/write operations
|
||||
+ VAD integration (skip silence)
|
||||
+ GPU detection in init()
|
||||
+ Improved error handling
|
||||
+ Context-aware processing
|
||||
+ Smaller SDL buffer (512 vs 1024)
|
||||
- Blocking queue operations
|
||||
- No VAD filtering
|
||||
- Poor error messages
|
||||
```
|
||||
|
||||
### main.cpp
|
||||
```diff
|
||||
+ Modern dark theme
|
||||
+ Custom button drawing (owner-draw)
|
||||
+ 30 FPS timer (was 20 FPS)
|
||||
+ Status text with GPU/CPU/threads
|
||||
+ DWM dark mode titlebar
|
||||
+ Improved layout handling
|
||||
+ Better font selection
|
||||
- Basic Windows theme
|
||||
- Standard buttons
|
||||
- Slow updates
|
||||
- Minimal status info
|
||||
```
|
||||
|
||||
### CMakeLists.txt
|
||||
```diff
|
||||
+ dwmapi library link
|
||||
+ Optimization flags (/O2 /GL /LTCG)
|
||||
+ Better include paths
|
||||
+ Separate Debug/Release outputs
|
||||
- Basic configuration
|
||||
```
|
||||
|
||||
### build.ps1
|
||||
```diff
|
||||
+ Complete rewrite
|
||||
+ GPU detection logic
|
||||
+ CUDA version checking
|
||||
+ Automatic DLL deployment
|
||||
+ Model download automation
|
||||
+ Colored output
|
||||
+ Error handling
|
||||
- CPU-only hardcoded
|
||||
- Manual deployment
|
||||
- No GPU support
|
||||
```
|
||||
|
||||
## Performance Impact
|
||||
|
||||
### Before
|
||||
- **Audio Loss**: Frequent dropped chunks
|
||||
- **CPU Usage**: 1-2 cores (~8%)
|
||||
- **GPU Usage**: 0%
|
||||
- **Latency**: 3-5 seconds
|
||||
- **UI FPS**: ~10-15 (choppy)
|
||||
- **Buffer Issues**: Frequent overruns
|
||||
|
||||
### After (CPU-Only)
|
||||
- **Audio Loss**: Zero (ring buffer)
|
||||
- **CPU Usage**: 24 cores (~60-80% during speech)
|
||||
- **GPU Usage**: 0% (incompatible CUDA version)
|
||||
- **Latency**: 2-3 seconds
|
||||
- **UI FPS**: 30 (smooth)
|
||||
- **Buffer Management**: Handles 30s bursts
|
||||
|
||||
### Potential (With GPU)
|
||||
- **Audio Loss**: Zero
|
||||
- **CPU Usage**: <10%
|
||||
- **GPU Usage**: 20-30%
|
||||
- **Latency**: <1 second
|
||||
- **Throughput**: >20x real-time
|
||||
|
||||
## Bug Fixes
|
||||
|
||||
1. ✅ Fixed audio chunk loss (ring buffer)
|
||||
2. ✅ Fixed CPU underutilization (thread count)
|
||||
3. ✅ Fixed UI glitches (proper timing)
|
||||
4. ✅ Fixed missing GPU support (detection)
|
||||
5. ✅ Fixed model loading errors (better paths)
|
||||
6. ✅ Fixed memory leaks (proper cleanup)
|
||||
7. ✅ Fixed race conditions (atomics)
|
||||
8. ✅ Fixed build issues (explicit GPU disable)
|
||||
|
||||
## Code Quality Improvements
|
||||
|
||||
- **Better error handling**: Graceful fallbacks
|
||||
- **More comments**: Explain complex logic
|
||||
- **Type safety**: size_t for sizes, proper casts
|
||||
- **Memory safety**: RAII, smart pointers ready
|
||||
- **Threading**: Atomic operations, no races
|
||||
- **Modularity**: Clear separation of concerns
|
||||
|
||||
## Configuration Changes
|
||||
|
||||
### WhisperConfig
|
||||
```cpp
|
||||
struct WhisperConfig {
|
||||
std::string model_path;
|
||||
std::string language = "en";
|
||||
int n_threads = std::thread::hardware_concurrency(); // NEW: 24
|
||||
int step_ms = 1500; // NEW: was 3000
|
||||
int length_ms = 8000; // NEW: was 10000
|
||||
bool use_gpu = true; // NEW: auto-detect
|
||||
int capture_id = 0;
|
||||
int n_gpu_layers = -1; // NEW: auto
|
||||
};
|
||||
```
|
||||
|
||||
## Testing Results
|
||||
|
||||
### Build
|
||||
- ✅ Clean compile on MSVC 2022
|
||||
- ✅ No linter errors
|
||||
- ✅ All warnings addressed
|
||||
- ✅ Proper DLL deployment
|
||||
|
||||
### Runtime (Expected)
|
||||
- ✅ Window opens correctly
|
||||
- ✅ Modern UI renders
|
||||
- ✅ Audio devices detected
|
||||
- ✅ Recording works
|
||||
- ✅ Transcription functions
|
||||
- ✅ No crashes
|
||||
- ✅ System tray works
|
||||
- ✅ Hotkey functions
|
||||
|
||||
## Known Limitations
|
||||
|
||||
1. **CUDA 11.7 Incompatible**: User has CUDA 11.7 but MSVC 2022 requires CUDA 12.4+
|
||||
- **Workaround**: Using optimized CPU-only build
|
||||
- **Solution**: Upgrade to CUDA 12.4+ for GPU support
|
||||
|
||||
2. **Single Language**: Currently English-only (base.en model)
|
||||
- **Workaround**: Use multilingual model (ggml-base.bin)
|
||||
|
||||
3. **Model in Binary**: Model path hardcoded in source
|
||||
- **Future**: UI-based model selection
|
||||
|
||||
## Upgrade Path
|
||||
|
||||
### To Enable GPU (CUDA)
|
||||
1. Download and install CUDA Toolkit 12.4+
|
||||
2. Clean build directory
|
||||
3. Run build script (will auto-detect new CUDA)
|
||||
4. Rebuild application
|
||||
|
||||
### To Enable GPU (Vulkan - Alternative)
|
||||
1. Download and install Vulkan SDK
|
||||
2. Set VULKAN_SDK environment variable
|
||||
3. Clean and rebuild
|
||||
|
||||
### To Use Different Model
|
||||
1. Download model from HuggingFace
|
||||
2. Place in `build/bin/Release/models/`
|
||||
3. Update `g_config.model_path` in main.cpp
|
||||
4. Rebuild
|
||||
|
||||
## Documentation Added
|
||||
|
||||
1. **README.md**: Complete user guide
|
||||
2. **CHANGES.md**: This detailed changelog
|
||||
3. **Code Comments**: Inline documentation
|
||||
4. **Build Output**: Informative messages
|
||||
|
||||
## Migration Notes
|
||||
|
||||
This is a **breaking change** from v1.0:
|
||||
- API compatible but implementation completely different
|
||||
- Rebuild required (not drop-in replacement)
|
||||
- Configuration values changed
|
||||
- UI completely redesigned
|
||||
|
||||
## Acknowledgments
|
||||
|
||||
- whisper.cpp team for the excellent base library
|
||||
- SDL2 for cross-platform audio
|
||||
- User feedback on performance issues
|
||||
- Added `src/timing.h` — timing model + progress estimator + persistence
|
||||
- Rewrote `src/main.cpp` — single-surface paint, widget model, animation clock, DPI
|
||||
- Rewrote `README.md`, `src/README.md`, `src/CHANGES.md`
|
||||
|
||||
---
|
||||
|
||||
**Version**: 2.0
|
||||
**Date**: November 26, 2025
|
||||
**Status**: Production Ready (CPU-only), GPU Ready (pending CUDA upgrade)
|
||||
|
||||
|
||||
|
||||
## v2.0 — Previous architecture (deprecated)
|
||||
|
||||
Used streaming with ring buffer, VAD, owner-draw child windows. Described in earlier versions of the docs.
|
||||
|
||||
Reference in New Issue
Block a user