Skip to main content

03-faster-whisper.md

kstack: book: Centerpoint Home Lab chapter: AI & Automation page: Faster-Whisper tags: [whisper, speech-to-text, ai, cuda, gpu, transcription]

Overview

Faster-Whisper is the speech-to-text transcription service for the homelab. It runs OpenAI's Whisper model via the faster-whisper library (CTranslate2 backend), which offers significantly faster inference than the original Whisper implementation at equivalent or lower VRAM usage.

It is used by N8N automation workflows for audio transcription tasks (e.g. podcast processing in the Minuspod pipeline).

Access

Type URL Notes
Internal https://whisper.home.local LAN access via Step-CA TLS

No external route — internal automation use only.

Configuration

Image: hwdsl2/whisper-server:cuda Compose project: ai-stack Runtime: nvidia CUDA Version: 12.9

Ports

Port Protocol Purpose
9015 TCP HTTP API (mapped from internal 9000)

Traefik Labels

traefik.enable: "true"
traefik.docker.network: traefik-net
traefik.http.routers.whisper.rule: Host(`whisper.home.local`)
traefik.http.routers.whisper.entrypoints: websecure
traefik.http.routers.whisper.tls.certresolver: step-ca
traefik.http.services.whisper.loadbalancer.server.port: 9000

Environment Variables

Variable Value Purpose
WHISPER_MODEL large-v3-turbo Whisper model variant to load
WHISPER_DEVICE cuda Run inference on GPU
WHISPER_COMPUTE_TYPE float16 FP16 precision (optimal for CUDA)
WHISPER_LANGUAGE en Default transcription language
NVIDIA_VISIBLE_DEVICES all Expose all NVIDIA GPUs
NVIDIA_DRIVER_CAPABILITIES compute,utility Required NVIDIA driver capabilities

Model: large-v3-turbo — the distilled variant of Whisper large-v3, offering near-large accuracy at roughly 3× the speed and reduced VRAM usage.

GPU Acceleration

runtime: nvidia
environment:
  - NVIDIA_VISIBLE_DEVICES=all
  - WHISPER_DEVICE=cuda
  - WHISPER_COMPUTE_TYPE=float16

Volumes / Bind Mounts

Host Path Container Path Purpose
/home/jeeves/docker/ai-stack/whisper/files /var/lib/whisper Transcription input/output files

Model weights are downloaded to a temp directory inside the container on first start and cached within the container layer (not persisted in a named volume).

Networks

Network Purpose
ai-stack_ai-internal Internal access from N8N workflows
traefik-net Exposes API via Traefik

API Usage

The whisper-server exposes a simple HTTP POST endpoint:

curl -X POST https://whisper.home.local/inference \
  -F file=@audio.mp3 \
  -F response_format=json

Response:

{"text": "Transcribed text here..."}

Dependencies

  • NVIDIA container runtime with RTX 5080 access
  • N8N (primary consumer via HTTP calls)

Notes / Gotchas

  • The large-v3-turbo model is downloaded on first container start. This can take several minutes and the container will appear unresponsive until the download completes.
  • float16 compute type requires a GPU with FP16 support. The RTX 5080 (Blackwell) supports this natively. On CPU-only fallback, use int8 instead.
  • The files bind mount (/var/lib/whisper) can be used to pre-stage audio files for batch transcription if needed.

Last Updated: 2026-06-16

Back to top