# CUDA 13.0 Setup Guide for win-dictation

## After Installing CUDA 13.0 Update 2

### Step 1: Verify Installation

Open a **NEW** PowerShell window (important - to get updated environment variables) and run:

```powershell
nvcc --version
```

You should see:
```
Cuda compilation tools, release 13.0, V13.0.xxx
```

### Step 2: Check Environment Variable

```powershell
$env:CUDA_PATH
```

Should output something like:
```
C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v13.0
```

If it's empty or points to v11.7, you need to:
1. Close **all** PowerShell/terminal windows
2. Reopen and check again

### Step 3: Clean Previous Build

```powershell
cd C:\code\whisper.cpp
Remove-Item -Recurse -Force build
```

### Step 4: Build with GPU Support

```powershell
powershell -ExecutionPolicy Bypass -File examples/win-dictation/build.ps1
```

You should see:
```
[1/6] Detecting GPU capabilities...
  [OK] NVIDIA GPU detected: NVIDIA GeForce RTX 3090
  [OK] CUDA 13.0 toolkit - EXCELLENT!
  [OK] Full GPU acceleration enabled
```

### Step 5: Verify GPU Build

After successful build, run:

```powershell
.\build\bin\Release\win-dictation.exe
```

In the application window, you should see:
```
Status: ⬤ Recording • GPU: ON • Buffer: X% • Threads: 24
```

The `GPU: ON` confirms it's using your RTX 3090!

## Expected Performance with GPU

### Before (CPU-only, 24 threads)
- Latency: 2-3 seconds
- CPU Usage: 60-80% during speech
- Throughput: ~5x real-time

### After (GPU - RTX 3090)
- Latency: **<1 second** 🚀
- CPU Usage: **<10%**
- GPU Usage: **20-30%**
- Throughput: **>20x real-time**

## Troubleshooting

### Build Still Shows CPU-Only

**Check CUDA_PATH:**
```powershell
$env:CUDA_PATH
```

If it's wrong, manually set it:
```powershell
$env:CUDA_PATH = "C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v13.0"
```

Then rebuild.

### CMake Can't Find CUDA

Make sure you installed:
- ✅ CUDA Toolkit 13.0 (not just drivers)
- ✅ Visual Studio integration components
- ✅ Development tools

Reinstall CUDA with "Custom" and ensure these are checked.

### Missing CUDA DLLs

If the app won't start after GPU build:

1. Check what's missing:
```powershell
ls "C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v13.0\bin\cudart64_*.dll"
```

2. Copy manually if needed:
```powershell
Copy-Item "C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v13.0\bin\cudart64_*.dll" build/bin/Release/
Copy-Item "C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v13.0\bin\cublas64_*.dll" build/bin/Release/
Copy-Item "C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v13.0\bin\cublasLt64_*.dll" build/bin/Release/
```

### GPU: OFF Even After GPU Build

This means the CUDA backend failed to initialize. Check:

1. **NVIDIA drivers up to date:**
```powershell
nvidia-smi
```
Driver version should be 520+

2. **CUDA runtime accessible:**
```powershell
Test-Path "C:\Windows\System32\nvcuda.dll"
```
Should be True

3. **Try different model:**
Some models don't support GPU well. Stick with base.en or larger.

## Testing GPU Performance

### Benchmark Test

1. Start recording
2. Speak continuously for 30 seconds
3. Watch Task Manager:
   - CPU should be <10%
   - GPU should show compute activity
   - GPU Memory should increase

### Compare CPU vs GPU

**CPU Build:**
```powershell
# Already built in build/bin/Release/
Measure-Command { .\build\bin\Release\win-dictation.exe }
```

**GPU Build:**
```powershell
# After CUDA 13 rebuild
Measure-Command { .\build\bin\Release\win-dictation.exe }
```

GPU should feel **much more responsive** with faster transcription.

## Advanced: Multiple CUDA Versions

If you need to keep CUDA 11.7 for other projects:

```powershell
# Build with specific CUDA version
$env:CUDA_PATH = "C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v13.0"
powershell -ExecutionPolicy Bypass -File examples/win-dictation/build.ps1
```

The build script will automatically find the highest version, but you can override.

## Expected Build Output with CUDA 13.0

```
=== Whisper Dictation Builder ===

[1/6] Detecting GPU capabilities...
  [OK] NVIDIA GPU detected: NVIDIA GeForce RTX 3090
  [OK] CUDA 13.0 toolkit - EXCELLENT!
  [OK] Full GPU acceleration enabled

[2/6] Setting up SDL2...
  [OK] SDL2 already configured

[3/6] Configuring CMake...
  Enabling CUDA backend...
-- CUDA Toolkit found
-- Using CUDA architectures: native (will use sm_86 for RTX 3090)
-- Configuring done
-- Generating done

[4/6] Building win-dictation...
-- Building CUDA files for sm_86
[OK] Build successful

[5/6] Deploying dependencies...
  [OK] Copied SDL2.dll
  Deploying CUDA runtime DLLs...
  [OK] Copied cudart64_130.dll
  [OK] Copied cublas64_13.dll
  [OK] Copied cublasLt64_13.dll

[6/6] Checking Whisper model...
  [OK] Model ready: base.en

=== Build Complete ===

Configuration:
  - GPU Backend: cuda
  - CUDA Version: 13.0
  - GPU: NVIDIA GeForce RTX 3090 (sm_86)
  - CPU Threads: 24 (backup)
  - Model: base.en

Run the application:
  C:\code\whisper.cpp/build/bin/Release\win-dictation.exe

GPU acceleration enabled! Expect 10-20x speedup! 🚀
```

## Notes

- The RTX 3090 has compute capability 8.6 (sm_86)
- CUDA 13.0 fully supports Ampere architecture
- You'll get optimal performance with this combination
- First run may be slower (CUDA kernel compilation/caching)

---

Once installed, just:
1. Close all terminals
2. Open new terminal
3. Run: `Remove-Item -Recurse -Force build; powershell -ExecutionPolicy Bypass -File examples/win-dictation/build.ps1`
4. Launch and enjoy GPU-accelerated transcription! 🎉








