This repository provides ComfyUI custom nodes for the advanced Fish Audio S2 Pro Text-to-Speech (TTS) model, enabling voice cloning, multi-speaker synthesis, and fine-grained prosody/emotion control within node-based workflows. It benefits ComfyUI users, researchers, and developers seeking realistic, emotionally rich, and highly controllable AI-generated speech.

How It Works

The nodes integrate the Fish Audio S2 Pro TTS model, a Dual-Autoregressive system trained on over 10 million hours of audio across 83 languages. Its core advantages include achieving naturalistic speech via reinforcement learning alignment and offering advanced features like zero-shot voice cloning from brief audio samples and inline emotion/prosody control using [tag] syntax. This approach enables sophisticated audio generation within a visual programming paradigm.

Quick Start & Requirements

Installation is recommended via ComfyUI Manager (search "FishAudioS2"). Manual setup involves cloning to ComfyUI/custom_nodes and running pip install -r requirements.txt. Key requirements include Python 3.10+, CUDA 11.8+ for GPU, and NVIDIA GPUs with 16GB VRAM (quantized) or 24GB+ (full precision). Models auto-download from HuggingFace.

Highlighted Details

• Zero-shot voice cloning from 10-30 second reference audio.
• Fine-grained control with 1500+ emotive tags (e.g., [excited], [whisper]).
• Full multilingual support for 83 languages, with automatic detection.
• Multi-speaker TTS synthesis and per-speaker audio isolation for lip-sync workflows.
• Performance optimizations: bf16/fp16/fp32 dtypes, SDPA, FlashAttention, SageAttention.

Maintenance & Community

The project links to the Fish Audio Discord server for community support. Specific details on maintainers, sponsorships, or a public roadmap are not explicitly detailed in the README.

Licensing & Compatibility

The project and model weights use the Fish Audio Research License, permitting research and non-commercial use. Commercial applications require a separate license from Fish Audio.

Limitations & Caveats

CPU/MPS inference is experimental and slow. FP8 quantized models require specific Ada/Blackwell GPUs (e.g., RTX 4090/5090). Reference text transcription via Whisper (MTB) may need manual input due to compatibility issues. torch.compile is Linux-only and requires careful configuration. Dependency management for descript-audio-codec/descript-audiotools needs careful handling of protobuf versions in shared environments.