Summary

VoiceSculptor provides an instruct text-to-speech (TTS) solution for advanced voice design and cloning. It enables researchers and power users to generate and manipulate synthetic voices via natural language, offering fine-grained control over attributes like pitch and emotion, built on LLaSA and CosyVoice2.

How It Works

The system comprises voice design and voice cloning components. Voice design synthesizes novel timbres from descriptive text, enhanced by RAG for command refinement and attribute control. Generated audio prompts CosyVoice2 for timbre cloning and speech synthesis.

Quick Start & Requirements

• Installation: Clone repo, create Python 3.10 Conda env, pip install -r requirements.txt.
• Models: Download VoiceSculptor-VD and xcodec2 via git lfs.
• Inference: Run python infer.py (specify model paths).
• RAG: Scripts provided for building/querying private text vector databases.
• WebUI: Launchable via python gradio/webui.py, supports vLLM.
• Prerequisites: Python 3.10, git lfs.

Highlighted Details

• Instruct TTS Eval: Benchmarks show VoiceSculptor achieving 67.6% average on InstructTTSEval (ZH: 85.4%), outperforming commercial TTS models in complex instruction following.
• RAG Integration: Supports custom private text vector databases for enhanced voice design.
• WebUI & vLLM: Released RAG and WebUI code, with WebUI supporting vLLM for accelerated inference.
• Demo: Official demo page and video available.

Maintenance & Community

Recent releases include RAG/WebUI/vLLM code (Jan 2026), technical report (Jan 2026), and demo assets (Jan 2026). Contact: jingbin.hu@mail.nwpu.edu.cn, lxie@nwpu.edu.cn. A WeChat group is available for technical discussions.

Licensing & Compatibility

Licensed under Apache 2.0, permitting free use for academic research, education, and legitimate applications. A strict usage disclaimer prohibits unauthorized voice cloning, impersonation, fraud, deepfakes, or illegal activities, emphasizing that generated voices are synthetic and not tied to specific individuals.

Limitations & Caveats

Key components remain under development or are marked as TODOs, including text generation code, sample dataset, voice cloning code, and training code. The project emphasizes responsible AI use, with generated voices being synthetic outputs.