OpenBMB/VoxCPM: VoxCPM: Tokenizer-Free TTS for Context-Aware Speech Generation and True-to-Life Voice Cloning

VoxCPM: Tokenizer-Free TTS for Context-Aware Speech Generation and True-to-Life Voice Cloning - OpenBMB/VoxCPM

Summary

VoxCPM is an open-source Python library and model family for tokenizer-free text-to-speech. The current VoxCPM2 release is a 2B-parameter model trained on more than 2 million hours of multilingual speech, supporting 30 languages and producing 48 kHz audio. It ships with a Python API, a CLI, a web demo, and serving paths through vLLM-Omni and Nano-vLLM.

What it is useful for

VoxCPM fits teams that need a single TTS system covering multilingual synthesis, voice cloning, and voice design from one API. Documented use cases include:

• Synthesizing natural speech from text in any of 30 supported languages, including Chinese dialects such as Cantonese and Sichuanese.
• Creating a voice from a natural-language description (for example, gender, age, tone) with no reference audio, by wrapping the description in parentheses at the start of the text.
• Cloning a speaker from a short reference clip, optionally steering emotion or pace while preserving the original timbre.
• Producing higher-fidelity clones by passing both the reference audio and its transcript, which lets the model continue from the reference.
• Streaming audio chunks via generate_streaming for low-latency applications.

Deployment options in the README include a Python service, a CLI, an OpenAI-compatible HTTP endpoint via vLLM-Omni, and a C++ on-device binary via llama.cpp-omni.

How engineers can use it

The README documents a minimal setup:

1. Install: pip install voxcpm. Requires Python 3.10–3.12, PyTorch ≥ 2.5.0, and CUDA ≥ 12.0.
2. Load the model and generate audio:

from voxcpm import VoxCPM
import soundfile as sf

model = VoxCPM.from_pretrained("openbmb/VoxCPM2", load_denoiser=False)
wav = model.generate(
    text="VoxCPM2 is the current recommended release.",
    cfg_value=2.0, inference_timesteps=10, seed=42,
)
sf.write("demo.wav", wav, model.tts_model.sample_rate)

3. Use generate_streaming for chunked output, or the voxcpm CLI (voxcpm design, voxcpm clone, voxcpm batch) to run voice design, cloning, and batch jobs without writing code.
4. For production, the README points to vllm serve openbmb/VoxCPM2 --omni for an OpenAI-compatible /v1/audio/speech endpoint and nano-vllm-voxcpm for higher throughput.

Documented constraints: VoxCPM2 needs about 8 GB of VRAM, and the cited RTF is ~0.3 on an RTX 4090 (~0.13 with Nano-vLLM). The README notes that character-level timestamps are best-effort and derived from word alignment, and references a "Risks and Limitations" section for additional caveats that are not included in the visible documentation excerpt.