Add GPLv3 license, docs (README, manual, architecture review); binaries via Releases page
This commit is contained in:
Win Dictation Dev
+129
-235
@@ -1,244 +1,138 @@
|
||||
# Whisper Dictation - AI Voice to Text for Windows
|
||||
# Win Dictation — Voice to Text for Windows
|
||||
|
||||
A high-performance, real-time speech-to-text application for Windows using OpenAI's Whisper model.
|
||||
|
||||
|
||||
## ✨ Features
|
||||
A push-to-talk speech-to-text utility for Windows. Press a hotkey, speak, and your words land in whatever app you were just using — fully offline, powered by Whisper.
|
||||
|
||||
### Performance
|
||||
- **Multi-Core CPU Support**: Automatically uses all available CPU cores (24 threads detected)
|
||||
- **GPU Acceleration**: Auto-detects and uses CUDA, Vulkan, or Metal when available
|
||||
- **Ring Buffer Audio**: Zero audio loss with lock-free ring buffer implementation
|
||||
- **Optimized Processing**: AVX2/FMA instructions for maximum performance
|
||||
## Quick Download
|
||||
|
||||
### User Interface
|
||||
- **Modern Dark Theme**: Polished, professional interface
|
||||
- **Real-Time Monitoring**:
|
||||
- Live VU meter for audio levels
|
||||
- Buffer status indicator
|
||||
- GPU/CPU usage display
|
||||
- **Smooth Animations**: 30 FPS UI updates for responsive experience
|
||||
- **System Tray Integration**: Minimize to tray with hotkey support
|
||||
**[Download Latest Release (WinDictation.zip)](release/WinDictation.zip)**
|
||||
|
||||
### Audio Processing
|
||||
- **Voice Activity Detection (VAD)**: Automatically filters silence
|
||||
- **Continuous Recording**: Maintains context between segments
|
||||
- **Multiple Microphone Support**: Select from all available input devices
|
||||
- **16kHz Sample Rate**: Optimized for Whisper model
|
||||
Extract the ZIP file and run `win-dictation.exe`. The release includes all required DLLs and a Whisper model.
|
||||
|
||||
## 🚀 Quick Start
|
||||
|
||||
### Build
|
||||
|
||||
```powershell
|
||||
powershell -ExecutionPolicy Bypass -File examples/win-dictation/build.ps1
|
||||
```
|
||||
|
||||
The build script will:
|
||||
1. Detect your GPU capabilities (CUDA, Vulkan)
|
||||
2. Download and configure SDL2
|
||||
3. Build the application with optimal settings
|
||||
4. Download the Whisper model (base.en - 140MB)
|
||||
5. Deploy all required DLLs
|
||||
|
||||
### Run
|
||||
|
||||
```
|
||||
build/bin/Release/win-dictation.exe
|
||||
```
|
||||
|
||||
Or double-click the exe in the build output directory.
|
||||
|
||||
## 🎯 Usage
|
||||
|
||||
### Controls
|
||||
- **Start/Stop Recording**: Click button or press `Ctrl+Shift+R`
|
||||
- **Clear Text**: Click "Clear" button
|
||||
- **Change Microphone**: Select from dropdown (auto-restarts recording)
|
||||
- **Minimize**: Close window (minimizes to system tray)
|
||||
- **Exit**: Right-click tray icon → Exit
|
||||
|
||||
### Indicators
|
||||
- **Level**: Real-time audio input level
|
||||
- **Buffer**: Current audio buffer usage (0-100%)
|
||||
- **Status**: Shows GPU/CPU mode, recording state, thread count
|
||||
|
||||
## ⚙️ Technical Details
|
||||
|
||||
### Architecture
|
||||
|
||||
#### Ring Buffer Audio Capture
|
||||
- **Lock-Free Design**: Audio thread never blocks
|
||||
- **30-Second Buffer**: Handles burst processing without loss
|
||||
- **Atomic Operations**: Prevents race conditions
|
||||
|
||||
#### Processing Pipeline
|
||||
```
|
||||
Audio Input → Ring Buffer → VAD → Whisper Inference → Text Output
|
||||
```
|
||||
|
||||
1. **SDL Audio Capture**: 512-sample chunks at 16kHz
|
||||
2. **Ring Buffer**: Lock-free circular buffer
|
||||
3. **VAD Processing**: Filters silence before inference
|
||||
4. **Whisper Inference**: Multi-threaded with context overlap
|
||||
5. **Text Output**: Appended to UI in real-time
|
||||
|
||||
### Performance Optimizations
|
||||
|
||||
#### CPU Mode (Current Build)
|
||||
- All 24 CPU threads utilized
|
||||
- AVX2/FMA SIMD instructions
|
||||
- Optimized memory layout
|
||||
- Minimal context switching
|
||||
|
||||
#### GPU Mode (When Available)
|
||||
- CUDA 12.4+ or Vulkan SDK required
|
||||
- Automatic offloading to GPU
|
||||
- Faster inference times
|
||||
- Lower CPU usage
|
||||
|
||||
### Model
|
||||
|
||||
Currently using `ggml-base.en.bin`:
|
||||
- **Size**: 140 MB
|
||||
- **Parameters**: 74 million
|
||||
- **Languages**: English only (optimized)
|
||||
- **Speed**: ~5x real-time on CPU, >20x on GPU
|
||||
- **Accuracy**: Excellent for general speech
|
||||
|
||||
To use a different model, place it in `build/bin/Release/models/` and update the config in `main.cpp`.
|
||||
|
||||
## 🔧 Troubleshooting
|
||||
|
||||
### GPU Not Detected
|
||||
- **CUDA**: Install CUDA Toolkit 12.4 or newer (CUDA 13.0 recommended)
|
||||
- **See [CUDA-SETUP.md](CUDA-SETUP.md) for detailed installation guide**
|
||||
- **Vulkan**: Install Vulkan SDK
|
||||
- CPU-only mode still provides excellent performance with all cores
|
||||
|
||||
### After Installing CUDA 13.0
|
||||
See **[CUDA-SETUP.md](CUDA-SETUP.md)** for complete setup instructions including:
|
||||
- Verification steps
|
||||
- Clean rebuild process
|
||||
- Performance benchmarking
|
||||
- Troubleshooting GPU issues
|
||||
|
||||
### Audio Not Working
|
||||
- Check microphone permissions in Windows Settings
|
||||
- Verify correct device selected in dropdown
|
||||
- Test microphone in Windows Sound settings
|
||||
|
||||
### Poor Transcription Quality
|
||||
- Ensure microphone is close (6-12 inches)
|
||||
- Reduce background noise
|
||||
- Check VU meter shows green when speaking
|
||||
- Try a larger model (medium.en or large-v3-turbo)
|
||||
|
||||
### High CPU Usage
|
||||
- Normal during active transcription
|
||||
- Reduces during silence (VAD filtering)
|
||||
- Consider enabling GPU acceleration
|
||||
|
||||
## 📊 Performance Benchmarks
|
||||
|
||||
### CPU-Only (24 threads, base.en model)
|
||||
- **Latency**: ~2-3 seconds
|
||||
- **Throughput**: ~5x real-time
|
||||
- **CPU Usage**: 60-80% during speech
|
||||
- **Memory**: ~500 MB
|
||||
|
||||
### GPU-Accelerated (RTX 3090, base.en model)
|
||||
- **Latency**: <1 second
|
||||
- **Throughput**: >20x real-time
|
||||
- **GPU Usage**: 20-30%
|
||||
- **CPU Usage**: <10%
|
||||
- **Memory**: ~1 GB (VRAM)
|
||||
|
||||
## 🆕 Recent Improvements
|
||||
|
||||
### v2.0 (Current)
|
||||
- ✅ **Ring buffer** implementation - no more dropped audio
|
||||
- ✅ **Multi-core CPU** support - uses all available threads
|
||||
- ✅ **GPU auto-detection** - CUDA/Vulkan support
|
||||
- ✅ **Modern UI** - dark theme, smooth animations
|
||||
- ✅ **VAD integration** - skip silence for efficiency
|
||||
- ✅ **Better error handling** - graceful fallbacks
|
||||
- ✅ **Status indicators** - real-time monitoring
|
||||
- ✅ **Build script** - automated setup and deployment
|
||||
|
||||
### Previous Issues (Fixed)
|
||||
- ❌ Audio chunks lost between recording and processing
|
||||
- ❌ No GPU utilization
|
||||
- ❌ Only used 1-2 CPU cores
|
||||
- ❌ Slow, glitchy interface
|
||||
- ❌ No real-time feedback
|
||||
- ❌ Poor error messages
|
||||
|
||||
## 🎨 UI Features
|
||||
|
||||
### Modern Dark Theme
|
||||
- Background: `#202124`
|
||||
- Surface: `#292A2D`
|
||||
- Primary: `#8AB4F8` (Blue)
|
||||
- Success: `#81C995` (Green)
|
||||
- Text: `#E8EAED`
|
||||
|
||||
### Responsive Layout
|
||||
- Auto-resizes with window
|
||||
- Maintains proper spacing
|
||||
- Smooth transitions
|
||||
|
||||
### Visual Feedback
|
||||
- VU meter with color coding
|
||||
- Buffer status bar
|
||||
- GPU/CPU indicator
|
||||
- Thread count display
|
||||
|
||||
## 🔮 Future Enhancements
|
||||
|
||||
- [ ] Push-to-talk mode
|
||||
- [ ] Multiple language support
|
||||
- [ ] Punctuation model integration
|
||||
- [ ] Export to file (TXT, SRT)
|
||||
- [ ] Custom hotkey configuration
|
||||
- [ ] Noise reduction filter
|
||||
- [ ] Model switching in UI
|
||||
- [ ] Real-time word highlighting
|
||||
|
||||
## 📝 License
|
||||
|
||||
This example is part of the whisper.cpp project and follows the same license (MIT).
|
||||
|
||||
## 🤝 Contributing
|
||||
|
||||
Improvements welcome! The code is designed to be:
|
||||
- **Readable**: Clear structure and comments
|
||||
- **Maintainable**: Modular design
|
||||
- **Extensible**: Easy to add features
|
||||
- **Performant**: Optimized critical paths
|
||||
|
||||
## 💡 Tips
|
||||
|
||||
### For Best Results
|
||||
1. Use a quality microphone
|
||||
2. Position mic 6-12 inches from mouth
|
||||
3. Speak clearly and naturally
|
||||
4. Minimize background noise
|
||||
5. Keep buffer below 50% (adjust step_ms if needed)
|
||||
|
||||
### For Development
|
||||
- See `transcriber.h/cpp` for core logic
|
||||
- See `main.cpp` for UI implementation
|
||||
- Adjust parameters in `WhisperConfig` struct
|
||||
- Enable logging in `whisper_full_params`
|
||||
|
||||
## 📚 Resources
|
||||
|
||||
- [Whisper.cpp](https://github.com/ggerganov/whisper.cpp)
|
||||
- [Whisper Paper](https://arxiv.org/abs/2212.04356)
|
||||
- [Model Download](https://huggingface.co/ggerganov/whisper.cpp)
|
||||
- [CUDA Toolkit](https://developer.nvidia.com/cuda-downloads)
|
||||
- [Vulkan SDK](https://vulkan.lunarg.com/)
|
||||
For new releases, run `.\src\package.ps1` after building, then attach `dist\WinDictation.zip` to a [Gitea Release](https://docs.gitea.com/usage/packages/overview) on your instance.
|
||||
|
||||
---
|
||||
|
||||
**Built with ❤️ using whisper.cpp**
|
||||
## Documentation
|
||||
|
||||
- **[User Manual](win-dictation-user-manual.html)** — How to use every feature
|
||||
- **[Architecture & Engineering Review](win-dictation-architecture-engineering-review.html)** — Deep dive into the codebase for developers
|
||||
|
||||
---
|
||||
|
||||
## Features
|
||||
|
||||
### Performance
|
||||
- **Physical-core threading**: Uses one thread per physical core for efficient batch transcription
|
||||
- **CPU-only**: Optimised for ordinary laptops — no GPU required
|
||||
- **Self-calibrating progress**: Learns transcription speed per model and machine, delivering a smooth countdown
|
||||
|
||||
### User Interface
|
||||
- **Single-surface rendering**: No inter-window seams or hairlines — the entire UI is one painted surface
|
||||
- **Dark theme**: Calm, elevated card design with hover/press feedback
|
||||
- **Per-monitor DPI awareness**: Looks sharp at any display scale
|
||||
- **System tray**: Minimise to tray, global hotkey to record
|
||||
|
||||
### Audio Processing
|
||||
- **Push-to-talk**: Press Ctrl+Shift+Space, speak, press again to transcribe
|
||||
- **Multiple microphones**: Select from all available input devices
|
||||
- **16 kHz sample rate**: Optimised for Whisper
|
||||
- **Silence trimming**: Leading and trailing silence is trimmed before transcription
|
||||
|
||||
## Quick Start
|
||||
|
||||
1. Launch `win-dictation.exe`
|
||||
2. Click into wherever you want text, then press `Ctrl+Shift+Space`
|
||||
3. Speak, then press `Ctrl+Shift+Space` again
|
||||
4. Your words appear in the app you were using
|
||||
|
||||
Full instructions in the [User Manual](win-dictation-user-manual.html).
|
||||
|
||||
### Controls
|
||||
- **Record/Stop**: Click the pill or press `Ctrl+Shift+Space`
|
||||
- **Hide window**: `Ctrl+Shift+H`
|
||||
- **Pin**: Keep window always-on-top
|
||||
- **Copy / Paste / Clear**: Text actions below the transcript
|
||||
- **Model / Mic / History**: Select from popup menus
|
||||
|
||||
## Building from Source
|
||||
|
||||
### Prerequisites
|
||||
|
||||
- **Windows 10/11**
|
||||
- **CMake** 3.5+
|
||||
- **Visual Studio 2022** with C++ workload
|
||||
- **SDL2** (included in deps/)
|
||||
|
||||
### Build Steps
|
||||
|
||||
```powershell
|
||||
cmake -S . -B build -G "Visual Studio 17 2022" ^
|
||||
-DSDL2_DIR="deps/SDL2-2.28.5/cmake"
|
||||
cmake --build build --config Release
|
||||
```
|
||||
|
||||
The executable will be at `build\bin\Release\win-dictation.exe`.
|
||||
|
||||
## Technical Details
|
||||
|
||||
### Architecture
|
||||
|
||||
```
|
||||
Hotkey → SDL Capture → Stop → Batch whisper_full → Text → Auto-paste
|
||||
```
|
||||
|
||||
1. **SDL audio capture**: 16 kHz mono recording into memory. CPU stays near idle while recording.
|
||||
2. **Stop-and-transcribe**: One `whisper_full` call processes the full clip at once.
|
||||
3. **Progress estimation**: Decayed online least-squares model per machine/model, fused with whisper's chunk progress into a strictly monotonic countdown.
|
||||
4. **Text output**: Inserted at cursor with smart spacing, copied to clipboard, and optionally auto-pasted into the window you came from.
|
||||
|
||||
### Models
|
||||
|
||||
Place `.bin` files in `models/` next to the executable, or download them from the Settings screen in-app:
|
||||
|
||||
| Model | Size | Best for |
|
||||
|-------|------|----------|
|
||||
| tiny.en | ~75 MB | Fastest — everyday dictation |
|
||||
| tiny.en-q8_0 | ~42 MB | Same speed, smaller file |
|
||||
| base.en-q5_1 | ~59 MB | Good accuracy bump for little cost |
|
||||
| base.en | ~142 MB | More accurate; still reasonable on two cores |
|
||||
| small.en-q5_1 | ~182 MB | Accurate, but slower |
|
||||
| small.en | ~466 MB | Most accurate — and slowest |
|
||||
|
||||
## Project Structure
|
||||
|
||||
```
|
||||
win-dictation/
|
||||
├── src/ # Application source
|
||||
│ ├── main.cpp # Window, painting, interaction, settings, clipboard, popups
|
||||
│ ├── transcriber.* # SDL capture, Whisper preload/inference, progress callbacks
|
||||
│ ├── timing.h # Learned timing model + live progress estimator
|
||||
│ ├── history.h # Session text files, index, pruning
|
||||
│ ├── downloader.h # WinHTTP model downloader (background thread)
|
||||
│ ├── stats.h # Lifetime usage totals + derived figures
|
||||
│ ├── settings.h # INI persistence
|
||||
│ ├── text_util.h # Transcript concatenation helpers
|
||||
│ ├── logging.h # Timestamped file log
|
||||
│ └── tests/ # Unit tests (test-core.exe)
|
||||
├── whisper/ # Whisper.cpp library
|
||||
├── ggml/ # GGML tensor library
|
||||
├── models/ # Whisper model files
|
||||
├── history/ # Saved dictation sessions
|
||||
├── release/ # Pre-built package
|
||||
└── CMakeLists.txt # Build configuration
|
||||
```
|
||||
|
||||
## License
|
||||
|
||||
[GNU General Public License v3.0](LICENSE)
|
||||
|
||||
This project is licensed under GPLv3. It incorporates [whisper.cpp](https://github.com/ggerganov/whisper.cpp) (MIT), [ggml](https://github.com/ggerganov/ggml) (MIT), and [SDL2](https://www.libsdl.org/) (zlib).
|
||||
|
||||
## Resources
|
||||
|
||||
- [User Manual](win-dictation-user-manual.html)
|
||||
- [Architecture & Engineering Review](win-dictation-architecture-engineering-review.html)
|
||||
- [Whisper.cpp](https://github.com/ggerganov/whisper.cpp)
|
||||
- [Model Downloads](https://huggingface.co/ggerganov/whisper.cpp)
|
||||
|
||||
Reference in New Issue
Block a user