296 lines
9.1 KiB
Markdown
296 lines
9.1 KiB
Markdown
# Win Dictation - AI Voice to Text for Windows
|
|
|
|
A high-performance, real-time speech-to-text application for Windows using OpenAI's Whisper model. Convert your voice to text with GPU acceleration support and a modern, user-friendly interface.
|
|
|
|
|
|
|
|
## 🚀 Quick Download
|
|
|
|
**[Download Latest Release (WinDictation.zip)](release/WinDictation.zip)**
|
|
|
|
Simply extract the ZIP file and run `win-dictation.exe`. The release includes all required DLLs and the Whisper model.
|
|
|
|
---
|
|
|
|
## ✨ Features
|
|
|
|
### Performance
|
|
- **Multi-Core CPU Support**: Automatically uses all available CPU cores for maximum performance
|
|
- **GPU Acceleration**: Auto-detects and uses CUDA or Vulkan when available
|
|
- **Smart Model Selection**: Automatically selects optimal model (tiny.en for CPU-only, base.en for GPU) for best performance
|
|
- **Ring Buffer Audio**: Zero audio loss with lock-free ring buffer implementation
|
|
- **Optimized Processing**: AVX2/FMA instructions for maximum performance
|
|
|
|
### User Interface
|
|
- **Modern Dark Theme**: Polished, professional interface
|
|
- **Real-Time Monitoring**:
|
|
- Live VU meter for audio levels
|
|
- Buffer status indicator
|
|
- GPU/CPU usage display
|
|
- **Smooth Animations**: 30 FPS UI updates for responsive experience
|
|
- **System Tray Integration**: Minimize to tray with hotkey support
|
|
|
|
### Audio Processing
|
|
- **Voice Activity Detection (VAD)**: Automatically filters silence
|
|
- **Continuous Recording**: Maintains context between segments
|
|
- **Multiple Microphone Support**: Select from all available input devices
|
|
- **16kHz Sample Rate**: Optimized for Whisper model
|
|
|
|
## 🎯 Usage
|
|
|
|
### Controls
|
|
- **Start/Stop Recording**: Click button or press `Ctrl+Shift+R`
|
|
- **Clear Text**: Click "Clear" button
|
|
- **Change Microphone**: Select from dropdown (auto-restarts recording)
|
|
- **Minimize**: Close window (minimizes to system tray)
|
|
- **Exit**: Right-click tray icon → Exit
|
|
|
|
### Indicators
|
|
- **Level**: Real-time audio input level
|
|
- **Buffer**: Current audio buffer usage (0-100%)
|
|
- **Status**: Shows GPU/CPU mode, recording state, thread count
|
|
|
|
## 🔨 Building from Source
|
|
|
|
### Prerequisites
|
|
|
|
- **Windows 10/11**
|
|
- **CMake** (3.5 or newer)
|
|
- **C++ Compiler** (MSVC 2019+ or MinGW)
|
|
- **PowerShell** (for build script)
|
|
- **Optional**: CUDA 12.4+ or Vulkan SDK (for GPU acceleration)
|
|
|
|
### Build Steps
|
|
|
|
1. **Clone the repository:**
|
|
```powershell
|
|
git clone <repository-url>
|
|
cd win-dictation
|
|
```
|
|
|
|
2. **Run the build script:**
|
|
```powershell
|
|
powershell -ExecutionPolicy Bypass -File src/build.ps1
|
|
```
|
|
|
|
The build script will:
|
|
- Detect your GPU capabilities (CUDA, Vulkan)
|
|
- Download and configure SDL2 automatically
|
|
- Build the application with optimal settings
|
|
- Download both Whisper models (tiny.en for CPU, base.en for GPU)
|
|
- Deploy all required DLLs
|
|
|
|
3. **Run the application:**
|
|
```powershell
|
|
build\bin\Release\win-dictation.exe
|
|
```
|
|
|
|
### Manual Build (Alternative)
|
|
|
|
If you prefer to build manually:
|
|
|
|
```powershell
|
|
# Configure CMake
|
|
cmake -B build -DWHISPER_SDL2=ON
|
|
|
|
# For GPU support (CUDA):
|
|
cmake -B build -DWHISPER_SDL2=ON -DGGML_CUDA=ON
|
|
|
|
# For GPU support (Vulkan):
|
|
cmake -B build -DWHISPER_SDL2=ON -DGGML_VULKAN=ON
|
|
|
|
# Build
|
|
cmake --build build --config Release
|
|
|
|
# The executable will be at: build\bin\Release\win-dictation.exe
|
|
```
|
|
|
|
### SDL2 Setup
|
|
|
|
The build script automatically downloads SDL2. If building manually, you can:
|
|
|
|
1. Download SDL2 from: https://github.com/libsdl-org/SDL/releases
|
|
2. Extract to `SDL2-mingw/` directory
|
|
3. Set `SDL2_DIR` in CMake to point to the SDL2 cmake directory
|
|
|
|
## ⚙️ Technical Details
|
|
|
|
### Architecture
|
|
|
|
#### Ring Buffer Audio Capture
|
|
- **Lock-Free Design**: Audio thread never blocks
|
|
- **30-Second Buffer**: Handles burst processing without loss
|
|
- **Atomic Operations**: Prevents race conditions
|
|
|
|
#### Processing Pipeline
|
|
```
|
|
Audio Input → Ring Buffer → VAD → Whisper Inference → Text Output
|
|
```
|
|
|
|
1. **SDL Audio Capture**: 512-sample chunks at 16kHz
|
|
2. **Ring Buffer**: Lock-free circular buffer
|
|
3. **VAD Processing**: Filters silence before inference
|
|
4. **Whisper Inference**: Multi-threaded with context overlap
|
|
5. **Text Output**: Appended to UI in real-time
|
|
|
|
### Performance Optimizations
|
|
|
|
#### CPU Mode
|
|
- All available CPU threads utilized
|
|
- AVX2/FMA SIMD instructions
|
|
- Optimized memory layout
|
|
- Minimal context switching
|
|
|
|
#### GPU Mode (When Available)
|
|
- CUDA 12.4+ or Vulkan SDK required
|
|
- Automatic offloading to GPU
|
|
- Faster inference times
|
|
- Lower CPU usage
|
|
|
|
### Model
|
|
|
|
The app automatically selects the optimal model based on your system:
|
|
|
|
**CPU-Only Systems:**
|
|
- Uses `ggml-tiny.en.bin` (75 MB)
|
|
- **Parameters**: 39 million
|
|
- **Speed**: ~10-15x real-time on CPU
|
|
- **Accuracy**: Good for general speech
|
|
- Optimized for slower machines
|
|
|
|
**GPU-Accelerated Systems:**
|
|
- Uses `ggml-base.en.bin` (140 MB)
|
|
- **Parameters**: 74 million
|
|
- **Speed**: >20x real-time on GPU
|
|
- **Accuracy**: Excellent for general speech
|
|
- Better accuracy with GPU acceleration
|
|
|
|
Both models are English-only (optimized). The app detects GPU availability at startup and selects the appropriate model automatically. To use a different model, place it in `models/` directory and the app will detect it.
|
|
|
|
## 📊 Performance Benchmarks
|
|
|
|
### CPU-Only (24 threads, tiny.en model)
|
|
- **Latency**: ~1-2 seconds
|
|
- **Throughput**: ~10-15x real-time
|
|
- **CPU Usage**: 40-60% during speech
|
|
- **Memory**: ~200 MB
|
|
- **Model**: Automatically selected for CPU-only systems
|
|
|
|
### CPU-Only (24 threads, base.en model - if manually selected)
|
|
- **Latency**: ~2-3 seconds
|
|
- **Throughput**: ~5x real-time
|
|
- **CPU Usage**: 60-80% during speech
|
|
- **Memory**: ~500 MB
|
|
|
|
### GPU-Accelerated (RTX 3090, base.en model)
|
|
- **Latency**: <1 second
|
|
- **Throughput**: >20x real-time
|
|
- **GPU Usage**: 20-30%
|
|
- **CPU Usage**: <10%
|
|
- **Memory**: ~1 GB (VRAM)
|
|
- **Model**: Automatically selected for GPU systems
|
|
|
|
## 🔧 Troubleshooting
|
|
|
|
### GPU Not Detected
|
|
- **CUDA**: Install CUDA Toolkit 12.4 or newer (CUDA 13.0 recommended)
|
|
- See `src/CUDA-SETUP.md` for detailed installation guide
|
|
- **Vulkan**: Install Vulkan SDK
|
|
- CPU-only mode still provides excellent performance with all cores
|
|
|
|
### Audio Not Working
|
|
- Check microphone permissions in Windows Settings
|
|
- Verify correct device selected in dropdown
|
|
- Test microphone in Windows Sound settings
|
|
|
|
### Poor Transcription Quality
|
|
- Ensure microphone is close (6-12 inches)
|
|
- Reduce background noise
|
|
- Check VU meter shows green when speaking
|
|
- Try a larger model (medium.en or large-v3-turbo)
|
|
|
|
### High CPU Usage
|
|
- Normal during active transcription
|
|
- Reduces during silence (VAD filtering)
|
|
- Consider enabling GPU acceleration
|
|
|
|
## 📁 Project Structure
|
|
|
|
```
|
|
win-dictation/
|
|
├── src/ # Main application source code
|
|
│ ├── main.cpp # UI and Windows message handling
|
|
│ ├── transcriber.* # Core transcription logic
|
|
│ └── build.ps1 # Automated build script
|
|
├── whisper/ # Whisper.cpp library
|
|
├── ggml/ # GGML tensor library
|
|
├── common/ # Shared utilities
|
|
├── models/ # Whisper model files
|
|
├── release/ # Pre-built release package
|
|
│ └── WinDictation.zip
|
|
└── CMakeLists.txt # Main build configuration
|
|
```
|
|
|
|
## 🆕 Recent Improvements
|
|
|
|
### v2.0 (Current)
|
|
- ✅ **Ring buffer** implementation - no more dropped audio
|
|
- ✅ **Multi-core CPU** support - uses all available threads
|
|
- ✅ **GPU auto-detection** - CUDA/Vulkan support
|
|
- ✅ **Modern UI** - dark theme, smooth animations
|
|
- ✅ **VAD integration** - skip silence for efficiency
|
|
- ✅ **Better error handling** - graceful fallbacks
|
|
- ✅ **Status indicators** - real-time monitoring
|
|
- ✅ **Build script** - automated setup and deployment
|
|
|
|
## 🔮 Future Enhancements
|
|
|
|
- [ ] Push-to-talk mode
|
|
- [ ] Multiple language support
|
|
- [ ] Punctuation model integration
|
|
- [ ] Export to file (TXT, SRT)
|
|
- [ ] Custom hotkey configuration
|
|
- [ ] Noise reduction filter
|
|
- [ ] Model switching in UI
|
|
- [ ] Real-time word highlighting
|
|
|
|
## 📝 License
|
|
|
|
This project uses the MIT license, following the same license as [whisper.cpp](https://github.com/ggerganov/whisper.cpp).
|
|
|
|
## 🤝 Contributing
|
|
|
|
Improvements welcome! The code is designed to be:
|
|
- **Readable**: Clear structure and comments
|
|
- **Maintainable**: Modular design
|
|
- **Extensible**: Easy to add features
|
|
- **Performant**: Optimized critical paths
|
|
|
|
## 📚 Resources
|
|
|
|
- [Whisper.cpp](https://github.com/ggerganov/whisper.cpp) - Core library
|
|
- [Whisper Paper](https://arxiv.org/abs/2212.04356) - Research paper
|
|
- [Model Download](https://huggingface.co/ggerganov/whisper.cpp) - Additional models
|
|
- [CUDA Toolkit](https://developer.nvidia.com/cuda-downloads) - GPU acceleration
|
|
- [Vulkan SDK](https://vulkan.lunarg.com/) - Alternative GPU backend
|
|
|
|
## 💡 Tips
|
|
|
|
### For Best Results
|
|
1. Use a quality microphone
|
|
2. Position mic 6-12 inches from mouth
|
|
3. Speak clearly and naturally
|
|
4. Minimize background noise
|
|
5. Keep buffer below 50% (adjust step_ms if needed)
|
|
|
|
### For Development
|
|
- See `src/transcriber.h/cpp` for core logic
|
|
- See `src/main.cpp` for UI implementation
|
|
- Adjust parameters in `WhisperConfig` struct
|
|
- Enable logging in `whisper_full_params`
|
|
|
|
---
|
|
|
|
**Built with ❤️ using whisper.cpp**
|
|
|