# Faster Whisper

## Overview

Faster-Whisper is the speech-to-text transcription service for the homelab. It
runs OpenAI's Whisper model via the `faster-whisper` library (CTranslate2 backend),
which offers significantly faster inference than the original Whisper implementation
at equivalent or lower VRAM usage.

It is used by N8N automation workflows for audio transcription tasks (e.g. podcast
processing in the Minuspod pipeline).

## Access

| Type      | URL                             | Notes                              |
|-----------|---------------------------------|------------------------------------|
| Internal  | `https://whisper.home.local`    | LAN access via Step-CA TLS         |

No external route — internal automation use only.

## Configuration

**Image:** `hwdsl2/whisper-server:cuda`
**Compose project:** `ai-stack`
**Runtime:** `nvidia`
**CUDA Version:** 12.9

### Ports

| Port   | Protocol | Purpose                              |
|--------|----------|--------------------------------------|
| `9015` | TCP      | HTTP API (mapped from internal `9000`) |

### Traefik Labels

```yaml
traefik.enable: "true"
traefik.docker.network: traefik-net
traefik.http.routers.whisper.rule: Host(`whisper.home.local`)
traefik.http.routers.whisper.entrypoints: websecure
traefik.http.routers.whisper.tls.certresolver: step-ca
traefik.http.services.whisper.loadbalancer.server.port: 9000
```

### Environment Variables

| Variable                     | Value              | Purpose                               |
|------------------------------|--------------------|---------------------------------------|
| `WHISPER_MODEL`              | `large-v3-turbo`   | Whisper model variant to load         |
| `WHISPER_DEVICE`             | `cuda`             | Run inference on GPU                  |
| `WHISPER_COMPUTE_TYPE`       | `float16`          | FP16 precision (optimal for CUDA)     |
| `WHISPER_LANGUAGE`           | `en`               | Default transcription language        |
| `NVIDIA_VISIBLE_DEVICES`     | `all`              | Expose all NVIDIA GPUs                |
| `NVIDIA_DRIVER_CAPABILITIES` | `compute,utility`  | Required NVIDIA driver capabilities   |

**Model:** `large-v3-turbo` — the distilled variant of Whisper large-v3, offering
near-large accuracy at roughly 3× the speed and reduced VRAM usage.

### GPU Acceleration

```yaml
runtime: nvidia
environment:
  - NVIDIA_VISIBLE_DEVICES=all
  - WHISPER_DEVICE=cuda
  - WHISPER_COMPUTE_TYPE=float16
```

## Volumes / Bind Mounts

| Host Path                                     | Container Path        | Purpose                    |
|-----------------------------------------------|-----------------------|----------------------------|
| `/home/jeeves/docker/ai-stack/whisper/files`  | `/var/lib/whisper`    | Transcription input/output files |

Model weights are downloaded to a temp directory inside the container on first
start and cached within the container layer (not persisted in a named volume).

## Networks

| Network                 | Purpose                             |
|-------------------------|-------------------------------------|
| `ai-stack_ai-internal`  | Internal access from N8N workflows  |
| `traefik-net`           | Exposes API via Traefik             |

## API Usage

The whisper-server exposes a simple HTTP POST endpoint:

```bash
curl -X POST https://whisper.home.local/inference \
  -F file=@audio.mp3 \
  -F response_format=json
```

Response:

```json
{"text": "Transcribed text here..."}
```

## Dependencies

- NVIDIA container runtime with RTX 5080 access
- N8N (primary consumer via HTTP calls)

## Notes / Gotchas

- The `large-v3-turbo` model is downloaded on first container start. This can take
  several minutes and the container will appear unresponsive until the download
  completes.
- `float16` compute type requires a GPU with FP16 support. The RTX 5080 (Blackwell)
  supports this natively. On CPU-only fallback, use `int8` instead.
- The `files` bind mount (`/var/lib/whisper`) can be used to pre-stage audio files
  for batch transcription if needed.

---
*Last Updated: 2026-06-16*