Whisper

Whisper

Purpose

• Use this skill for local speech-to-text transcription and subtitle generation.
• Prefer it when privacy, offline processing, or batch audio workflows matter.
• Whisper works well for meetings, podcasts, interviews, voice notes, and extracted video audio.
• It supports multilingual transcription and language detection.

Install

• Standard Whisper package:

pip install openai-whisper

• Faster inference alternative:

pip install faster-whisper

• faster-whisper is often the better default for production batch jobs because it is typically faster at similar accuracy.

Command Line Usage

• Basic CLI transcription:

whisper audio.mp3 --model medium --language en

• Use GPU explicitly:

whisper audio.mp3 --model medium --language en --device cuda

• Generate subtitles in SRT format:

whisper audio.mp3 --model medium --language en --output_format srt

• The CLI is a good fit for one-off transcription, shell scripts, and media preprocessing jobs.

Python API

import whisper

model = whisper.load_model("medium")
result = model.transcribe("audio.mp3")

print(result["text"])
print(result["language"])
print(result["segments"][:2])

• Standard load pattern:
• import whisper
• model = whisper.load_model('medium')

What transcribe() Returns

• text: the full transcript

• segments: timestamped segment-level outputs

• language: detected or selected language code

• Example shape:

{
    "text": "Full transcript text",
    "language": "en",
    "segments": [
        {"id": 0, "start": 0.0, "end": 4.5, "text": "Hello everyone"},
    ],
}

Model Sizes

• tiny

• base

• small

• medium

• large-v3

• The tradeoff is simple:

• smaller models are faster and cheaper

• larger models are slower but more accurate

Model Selection Guidance

• tiny: fast experiments and low-resource CPU runs
• base: simple automation on clean audio
• small: balanced for lightweight production tasks
• medium: common quality default for serious transcription
• large-v3: best accuracy when latency and VRAM are acceptable

Language Selection

• Set the language when you know it:

whisper audio.mp3 --model medium --language en

• Explicit language hints usually improve speed and stability.
• If the language is unknown, let the model detect it.

Language Detection

• Whisper can estimate the spoken language from audio features.
• Example pattern:

import whisper

model = whisper.load_model("medium")
audio = whisper.load_audio("audio.mp3")
audio = whisper.pad_or_trim(audio)
mel = whisper.log_mel_spectrogram(audio).to(model.device)

_, probs = model.detect_language(mel)
language = max(probs, key=probs.get)
print(language)

• model.detect_language(audio) is the key workflow concept, though in practice you pass the processed spectrogram tensor.

Subtitle Generation

• Generate .srt subtitles from the CLI:

whisper audio.mp3 --model medium --language en --output_format srt

• Subtitle outputs are useful for:
• video captions
• podcast transcripts
• lecture indexing
• searchable archives

Batch Processing Multiple Files

• Simple shell loop:

Get-ChildItem *.mp3 | ForEach-Object {
  whisper $_.FullName --model medium --language en --output_format srt
}

• Python batch example:

from pathlib import Path

import whisper

model = whisper.load_model("medium")

for path in Path("audio").glob("*.mp3"):
    result = model.transcribe(str(path))
    out_path = path.with_suffix(".txt")
    out_path.write_text(result["text"], encoding="utf-8")

• Batch processing is a common pattern for meeting folders, call archives, and downloaded media collections.

GPU vs CPU

• GPU example from the CLI:

whisper audio.mp3 --model medium --device cuda

• GPU is strongly preferred for:

• medium

• large-v3

• multi-file batch jobs

• CPU is acceptable for:

• tiny

• base

• occasional short clips

Common Use Cases

• YouTube audio transcription after extracting audio from video
• meeting notes from Zoom or Teams recordings
• podcast transcription for search and republishing
• lecture indexing and subtitle generation
• multilingual voice note transcription

faster-whisper

• Install with:

pip install faster-whisper

• It is often around 4x faster while maintaining comparable accuracy.
• It is a strong choice for MLOps pipelines where throughput matters.

faster-whisper Example

from faster_whisper import WhisperModel

model = WhisperModel("medium", device="cuda", compute_type="float16")
segments, info = model.transcribe("audio.mp3", beam_size=5)

print(info.language, info.language_probability)

for segment in segments:
    print(f"[{segment.start:.2f} -> {segment.end:.2f}] {segment.text}")

• faster-whisper is particularly useful for server-side or queue-based transcription systems.