Rhubarb Lip Sync is a command-line tool that automatically generates 2D mouth animations from voice recordings, ideal for game developers and animators. It analyzes audio to create lip-sync data, supporting various export formats and integrating with popular animation software.

How It Works

The tool analyzes audio files using either PocketSphinx (English) or a phonetic recognizer (language-independent) to identify speech sounds. It then maps these sounds to a set of predefined mouth shapes (6 basic, up to 3 optional extended shapes) based on phonetic analysis, generating timing data for animation.

Quick Start & Requirements

• Install by downloading the latest release for your OS from the releases page.
• Usage: rhubarb -o output.txt my-recording.wav
• Supported audio formats: WAVE (.wav), Ogg Vorbis (.ogg).
• For building from source: CMake 3.10+, C++17 compiler, Boost, JDK 8.x (for Spine build).

Highlighted Details

• Supports 6 basic and up to 3 extended mouth shapes, inspired by Hanna-Barbera and widely adopted standards.
• Integrates with Adobe After Effects, Moho, OpenToonz, Spine, and Vegas Pro.
• Offers multiple output formats: TSV, XML, and JSON for flexible integration.
• Includes a --machineReadable option for JSON-formatted progress and status messages, useful for automation.

Maintenance & Community

• Project activity and updates can be tracked via GitHub Actions CI.
• Feedback and bug reports can be submitted via GitHub Issues.
• Follow updates on Twitter.

Licensing & Compatibility

• The project does not explicitly state a license in the README.
• Compatibility for commercial use or closed-source linking is not specified.

Limitations & Caveats

• The PocketSphinx recognizer is limited to English audio.
• The README does not specify a license, which may impact commercial adoption.