VoxCPM 2¶

Release Date: March 2026
Parameter Size: 2B
Sampling Rate: 48kHz
Languages: 30 languages

Important

VoxCPM 2 is the current recommended release for new deployments and new feature work.

Overview¶

VoxCPM 2 is the latest major release — a 2B parameter model trained on 2.36 million hours of multilingual data. It represents a significant leap in capacity, quality, and controllability over the 1.x series.

Key characteristics:

48kHz audio output via AudioVAE V2 (asymmetric 16kHz encode → 48kHz decode)
30-language multilingual support
Voice Design: create a voice from natural-language description, no reference audio needed
Style Control: control emotion, pace, and speaking style of a cloned voice via text tags
Isolated reference channel for voice cloning (no matching transcript required)
Concat-Projection residual LM fusion and multi-token DiT conditioning for richer expressiveness
Built on a MiniCPM-4 backbone

Use VoxCPM 2 for all new projects. It is the recommended default for multilingual synthesis, voice cloning, voice design, and production deployment.

What’s New¶

🌍

30-Language Multilingual

Trained on 2.36 million hours of data (1.8M zh+en base + 560K multilingual), now covering 30 languages across multiple language families.

🎨

Voice Design & Style Control

Design a voice from scratch with natural language descriptions, or control the speaking style of a cloned voice — all through simple text tags.

🔊

48kHz Audio Output

A redesigned AudioVAE V2 with 3x higher upsampling ratio and sample-rate-conditioned decoding produces studio-quality 48kHz audio output.

🧠

Redesigned Fusion Architecture

Concat-Projection fusion and multi-token DiT conditioning replace additive shortcuts, preserving richer information flow throughout the pipeline.

Language Support¶

VoxCPM 2 supports 30 languages spanning diverse language families. Building on the original 1.8 million-hour Chinese and English corpus, we added 560,000 hours of multilingual data to enable high-quality synthesis across:

Language Family	Languages
East Asian	Chinese, Japanese, Korean
Southeast Asian	Burmese, Indonesian, Khmer, Lao, Malay, Tagalog, Thai, Vietnamese
South Asian	Hindi
European (Germanic)	Danish, Dutch, English, Finnish, German, Norwegian, Swedish
European (Romance)	French, Italian, Portuguese, Spanish
European (Other)	Greek, Polish, Russian, Turkish
Semitic	Arabic, Hebrew
African	Swahili

Architecture¶

VoxCPM 2 retains the four-stage pipeline of VoxCPM — Local Encoder → Text-Semantic LM → Residual Acoustic LM → Local DiT (CFM) — while redesigning three core information pathways for better capacity and expressiveness.

Feature Comparison¶

Feature	VoxCPM 1 / 1.5	VoxCPM 2
Patch Size	2 / 4	4
Residual LM Layers	6	8
FSQ Latent Dim	256	512
Max Sequence Length	4096	8192
AudioVAE Output	16kHz / 44.1kHz	48kHz
Encode / Decode Rate	Symmetric (same SR)	Asymmetric (16kHz -> 48kHz)
Residual LM Fusion	Additive	Concat + Projection
DiT Conditioning	Single token (add)	Multi-token (concat)
Reference Audio	Prompt continuation	Isolated ref channel
Languages	2 (zh, en)	30
Controllability	–	Voice Design + Style Control

Residual LM Fusion: Additive → Concat-Projection¶

In VoxCPM 1.x, the input to the Residual Acoustic LM is formed by adding the base LM output and the local encoder features. VoxCPM 2 replaces this with a concatenation followed by a learnable linear projection:

# VoxCPM 1.x
residual_input = lm_output + masked_audio_embed

# VoxCPM 2
residual_input = Linear₂ₕ→ₕ( concat(lm_output, masked_audio_embed) )

This gives the Residual LM more flexibility to learn how to combine semantic and acoustic information, rather than being constrained to element-wise addition.

DiT Conditioning: Single Token → Multi-Token Prefix¶

In VoxCPM 1.x, the LM hidden state and Residual LM hidden state are summed into a single conditioning vector, which is then added to the diffusion timestep embedding and fed to the DiT as one prefix token.

VoxCPM 2 instead concatenates the two projected hidden states (doubling the dimension), then reshapes them into multiple prefix tokens that are presented to the DiT alongside the timestep token:

# VoxCPM 1.x DiT input sequence:
[ (mu + t) | cond | x ]      ← 1 conditioning token

# VoxCPM 2 DiT input sequence:
[ mu₁ | mu₂ | t | cond | x ]  ← 2 conditioning tokens + timestep token

This allows the DiT’s attention mechanism to independently attend to semantic-level and acoustic-level information without information collapse from early fusion.

Isolated Reference Audio Channel¶

VoxCPM 1.x only supports voice cloning through prompt continuation (concatenating prompt audio with generation). VoxCPM 2 introduces a structurally isolated reference audio mechanism using dedicated special tokens:

[ <ref_start> | ref_audio_patches | <ref_end> | text_tokens | <audio_start> | generation... ]

This decouples the timbre reference from the continuation context, enabling four generation modes:

Zero-shot: No reference audio, synthesize from text only
Continuation: Prompt text + prompt audio for seamless continuation
Reference-only: Isolated voice cloning from a reference clip
Combined: Reference audio for timbre + prompt audio for context. We observe that this mode yields a slight improvement in voice cloning similarity compared to using reference or continuation alone.

AudioVAE V2: Native 48kHz with Sample-Rate Conditioning¶

The AudioVAE has been completely redesigned:

Asymmetric encode/decode design: Unlike v1/v1.5 where encoder and decoder operate at the same sample rate, V2 encodes at 16kHz (640x downsampling, keeping the LM token rate low at 6.25Hz) but decodes directly to 48kHz via a 1920x upsampling decoder. This achieves high-quality output without increasing the LM sequence length.
Decoder capacity: Channel width increased to 2048, with 6 upsampling stages [8, 6, 5, 2, 2, 2]
Sample-rate conditioning: A new SampleRateConditionLayer injects scale-bias modulation at each decoder block, allowing the same model to decode at different target sample rates

Controllable Generation¶

VoxCPM 2 introduces two new controllable generation features. Both use a simple convention: place control instructions inside parentheses () before the target text.

Voice Design¶

Create a voice from a natural language description without any reference audio. Simply describe the desired voice characteristics in parentheses:

from voxcpm import VoxCPM
import soundfile as sf

model = VoxCPM.from_pretrained("openbmb/VoxCPM2")

wav = model.generate(
   text="(A warm, gentle female voice in her 30s with a calm and soothing tone) "
        "Welcome to VoxCPM 2, the next generation of realistic speech synthesis.",
   cfg_value=2.0,
   inference_timesteps=10,
)
sf.write("voice_design.wav", wav, model.tts_model.sample_rate)

Tip

Voice Design works best with descriptive attributes such as age, gender, pitch, speaking pace, emotional tone, and vocal texture. Be as specific as you like — the model interprets natural language descriptions.

Style Control¶

Control the speaking style while using a reference audio for voice cloning. Pass control tags in parentheses alongside the reference audio:

from voxcpm import VoxCPM
import soundfile as sf

model = VoxCPM.from_pretrained("openbmb/VoxCPM2")

wav = model.generate(
   text="(Speaking slowly with a whispering, mysterious tone) "
        "The secret lies hidden in the ancient library, waiting to be discovered.",
   reference_wav_path="reference_speaker.wav",
   cfg_value=2.0,
   inference_timesteps=10,
)
sf.write("style_control.wav", wav, model.tts_model.sample_rate)

Note

In Style Control mode, the reference audio determines who speaks (timbre), while the text tag in parentheses controls how they speak (style, emotion, pace, etc.).

Usage Examples¶

For installation and the shared generate() API, start with Quick Start. The examples below focus on VoxCPM 2 specific capabilities.

Reference-Only Voice Cloning¶

wav = model.generate(
   text="This is a voice cloning demonstration using VoxCPM 2.",
   reference_wav_path="speaker_reference.wav",
   cfg_value=2.0,
   inference_timesteps=10,
)
sf.write("cloned.wav", wav, model.tts_model.sample_rate)

Multilingual Generation¶

# Korean
wav = model.generate(
   text="VoxCPM 2는 30개 언어를 지원하는 차세대 음성 합성 모델입니다.",
   reference_wav_path="korean_speaker.wav",
   cfg_value=2.0,
)
sf.write("korean.wav", wav, model.tts_model.sample_rate)

# French
wav = model.generate(
   text="VoxCPM 2 prend en charge la synthèse vocale en trente langues différentes.",
   reference_wav_path="french_speaker.wav",
   cfg_value=2.0,
)
sf.write("french.wav", wav, model.tts_model.sample_rate)

Migration Guide¶

From VoxCPM 1.5 to VoxCPM 2¶

Update Model Path: Point to VoxCPM2 checkpoint
Update Sample Rate: Prefer model.tts_model.sample_rate when saving audio (48000 for VoxCPM 2)
Voice Cloning API: Use the new reference_wav_path parameter for isolated voice cloning (prompt_wav_path still works for continuation mode)
Controllable Features: Explore Voice Design and Style Control by adding text tags in parentheses

Backward Compatibility¶

VoxCPM 1.0 and 1.5 models and configurations remain fully supported
Code automatically detects model architecture (voxcpm vs voxcpm2) from config.json
The generate() API is backward-compatible; new parameters are optional