This project provides a robust, modular, and self-hostable web backend for AI-powered video translation and dubbing, targeting users who need an efficient end-to-end solution for video localization. It aims to simplify the complex process of translating and dubbing videos, offering high-quality results and reducing manual effort.

How It Works

The system employs a microservices architecture, separating frontend requests, general video management, and GPU-intensive audio processing. It leverages Docker for deployment and Kubernetes for orchestration, ensuring scalability and resource optimization. The core workflow relies on a task queue (RabbitMQ) and workers for handling video processing, with a focus on using reliable and performant components like faster-whisper for transcription.

Quick Start & Requirements

• Installation: Deploy via Kubernetes (kubectl apply -k ./k8s/prod) or Docker Compose (docker compose up).
• Prerequisites: Python 3.9.19, PyTorch (GPU version required), FFmpeg, RabbitMQ. GPU with NVIDIA drivers is essential for the workload service.
• Setup: Local setup involves installing Python dependencies (pip install -r requirements.txt), configuring RabbitMQ, and potentially downloading faster-whisper models. Kubernetes/Docker Compose deployment is recommended for ease of use.
• Links: Online Demo, Grafana Monitoring, Frontend Repo, Offline Client

Highlighted Details

• Microservices architecture with dedicated GPU workload container.
• Supports self-hosting via Kubernetes and Docker Compose.
• Utilizes faster-whisper for efficient speech-to-text.
• Modular design allows for extensibility and secondary development.

Maintenance & Community

• Active development with a focus on reliability and user experience.
• Community support via a QQ group (ID: 536918174).
• Developer presence on Bilibili and X (formerly Twitter).

Licensing & Compatibility

• The README does not explicitly state a license. The project is presented as open-source and free to use. Compatibility for commercial use or closed-source linking is not specified.

Limitations & Caveats

• The GPU version of PyTorch is mandatory, with no readily available CPU fallback.
• Some TTS (Text-to-Speech) and translation models are still under evaluation or have stability issues.
• Local deployment instructions may be outdated; refer to Dockerfiles for current environment setup.