ViiTorVoice-NAR is a non-autoregressive speech generation system designed for high-fidelity voice cloning and precise local speech editing. It targets developers and researchers seeking advanced control over synthesized speech, offering low-latency inference and fine-grained manipulation capabilities. The system benefits users by enabling realistic voice replication and efficient audio post-production workflows.

How It Works

This system employs a non-autoregressive architecture, deployed via split gRPC v2 services with an HTTP gateway for simplified access. Core functionalities include voice cloning, where target speech is synthesized from prompt audio or a codebook, and local editing, which resynthesizes only the modified segments of an audio clip based on original and edited text. It also supports emotion and paralinguistic control through text tags and Classifier-Free Guidance (CFG) parameters, enhancing expressiveness. This approach facilitates low-latency inference, achieving approximately 60 ms first-frame latency.

Quick Start & Requirements

Setup involves running bash init_env.sh to create a Python virtual environment and install inference dependencies. Models are downloaded to the local_models/ directory using huggingface-cli download ZzWater/ViiTorVoice-NAR --local-dir local_models --local-dir-use-symlinks False, ensuring files are not symlinked. Services are managed with ./run_grpc_v2.sh (start, status, logs, stop). The HTTP service defaults to http://127.0.0.1:7861. Configuration for ports, model paths, and GPU settings is available in viitorvoice/grpc_server/deploy.env. Further details are in the HTTP API Usage and gRPC API Usage documentation.

Highlighted Details

• Low-Latency Inference: Achieves ~60 ms end-to-end first-frame latency.
• Flexible Voice Cloning: Supports synthesis from prompt audio or codebook inputs.
• Precise Local Editing: Enables targeted resynthesis of speech segments.
• Expressive Control: Integrates emotion and paralinguistic control via text tags and CFG.

Maintenance & Community

The provided README does not contain specific details regarding maintainers, community channels (e.g., Discord, Slack), or a public roadmap.

Licensing & Compatibility

The README does not specify a software license. Users should verify licensing terms before commercial use or integration into closed-source projects.

Limitations & Caveats

The README does not detail known limitations, alpha status, or specific bugs. The effectiveness of emotion and paralinguistic control is dependent on the availability of corresponding tags in the model's training data.