Automates subtitle synchronization with video content, addressing the common issue of misaligned captions. It offers a language-agnostic solution for video editors, content creators, and users needing precise subtitle timing, significantly reducing manual effort. A browser-based demo provides immediate, no-install access.

How It Works

The core approach discretizes both video audio and subtitle timing into 10ms windows. It then employs a Voice Activity Detector (VAD) on the audio and checks subtitle presence to create binary strings representing speech/subtitle activity. These strings are aligned using a Fast Fourier Transform (FFT) algorithm, achieving O(n log n) complexity for efficient and accurate temporal offset calculation.

Quick Start & Requirements

• Installation: Requires ffmpeg (ensure it's in your PATH). Install via pip install ffsubsync. Python version >= 3.6 is supported.
• Browser Demo: Available at https://smacke.github.io/ffsubsync for immediate use without installation.
• Docker: Prebuilt images available via docker pull ghcr.io/smacke/ffsubsync:latest.
• Dependencies: ffmpeg, Python >= 3.6. Optional dependencies for advanced VAD (torch).

Highlighted Details

• Language-agnostic subtitle synchronization.
• Robust automatic character encoding detection for diverse subtitle file formats.
• Offers a fully functional browser-based demo for quick, no-setup testing.
• Supports advanced synchronization tuning via numerous command-line flags (e.g., alternative VADs, multi-segment analysis).
• Can utilize Whisper.cpp for transcription-based synchronization when video audio is unavailable or problematic.

Maintenance & Community

The project originated from HackIllinois 2019. Development is supported via GitHub Sponsors. No specific community channels (Discord/Slack) or detailed roadmap are provided in the README.

Licensing & Compatibility

Licensed under the permissive MIT License. This license allows for commercial use and integration into closed-source projects without significant restrictions.

Limitations & Caveats

The tool may struggle with synchronization when video content contains breaks or segments (like recaps) that are absent in the corresponding subtitle files, or vice versa. Handling such structural discrepancies outside of the beginning/end segments is noted as future work.