Offline Voice

Offline voice - VieNeu-TTS and voice cloning

The system has two narration engines running side by side. They are deliberately not merged.

Keep both. Do not "simplify" by deleting one: a weak machine reads at roughly real time, which is unacceptable for a long script, while Gemini costs money on every preview click.

Why VieNeu and not F5-TTS / XTTS / Chatterbox

Checked against the requirement "Vietnamese + English, cloning, and shippable in an open-source repo":

• Chatterbox (MIT, excellent English) has no Vietnamese. Disqualified on language.
• F5-TTS code is MIT but the weights are CC-BY-NC, and every Vietnamese finetune is CC-BY-NC-SA. Disqualified on licence.
• viXTTS / XTTS-v2 inherit Coqui's CPML non-commercial licence. Disqualified on licence.
• VietTTS (dangvansam) is Apache-2.0 code but the installer is Linux-only, and its weights are CC-BY-NC too.
• VieNeu-TTS is Apache 2.0 on the code and on the v3 Turbo weights (pnnbao-ump/VieNeu-TTS-v3-Turbo, trained from scratch on ~10k hours, not a finetune), plus Apache 2.0 on its codec dependency OpenMOSS-Team/MOSS-Audio-Tokenizer-Nano. Vietnamese-first with English code-switching, torch-free on CPU via ONNX, clones from 3-8 seconds of reference audio.

If a future engine is evaluated, check the weights licence separately from the code licence. That is where every other candidate failed.

Licence trap inside the package itself. vieneu/assets/voices.json carries "license": "CC BY-NC 4.0" and "Model and voices are for non-commercial use only". That notice is scoped to that file, which is the v2 preset pack: 6 voices named Binh, Tuyen, Vinh, Doan, Ly, Ngoc, default Binh. We run mode="v3turbo", which reads voices_v3_turbo.json (14 Vietnamese voices, default Phạm Tuyên). The two sets have zero overlap - verified by comparing the key sets - so nothing on our path is NC-licensed. If anyone ever switches the engine mode away from v3turbo, or a preset named Binh/Tuyen/Vinh/Doan/Ly/Ngoc shows up in the picker, the commercial position changes and must be rechecked.

Install tiers - keep them separate

Two independent levels. Never collapse them into one check:

1. Speech only: pip install vieneu. Torch-free (ONNX Runtime), ~30 MB of wheels. This is enough for all 14 preset voices.
2. Cloning: additionally pip install torch torchaudio, 2-3 GB.

start/doctor.mjs reports these as vieneu and vieneu-clone, and only asks about torch once vieneu is present. Forcing a 2-3 GB torch download on someone who only wants offline narration is the mistake to avoid.

The first synthesis after a server start downloads ~1 GB of model from HuggingFace and takes 30s; subsequent process starts take ~15s to load. Always surface this in the UI, otherwise the first click looks like a hang.

Known issues

Symptom: Cloning fails with ImportError: TorchCodec is required for load_with_torchcodec, or RuntimeError: Could not load libtorchcodec. Cause: add_voice() reads the reference through torchaudio.load(). From torchaudio 2.9 every backend routes through torchcodec, which needs FFmpeg shared libraries - the usual Windows FFmpeg builds are static, so it can never load. Installing torchcodec does not fix it. Passing backend="soundfile" does not fix it either (verified: all three backends raise the same error). Fix: monkeypatch torchaudio.load to use soundfile before importing vieneu. soundfile is already a vieneu dependency, so this adds nothing to install. Guard it in try/except so that a machine without torch still gets preset voices.

import torch, torchaudio, soundfile as sf
def _load(uri, *a, **kw):
    data, sr = sf.read(str(uri), dtype="float32", always_2d=True)
    return torch.from_numpy(data.T).contiguous(), sr
torchaudio.load = _load

Symptom: Voices are filed under the wrong region, or every male voice looks southern. Cause: list_preset_voices() returns labels shaped Name — <Nam|Nữ> · <Bắc|Trung|Nam> · Phong cách <...>. Field 1 is gender, field 2 is region, and the word "Nam" means both "male" and "southern". Fix: parse strictly by position, never by searching for the substring "Nam". Regression cases: Thái Sơn = gender nam / region nam; Mai Anh = gender nu / region bac.