A Claude Code plugin transforms any video into a narrated recap with Chinese commentary, targeting users seeking automated video summarization. It simplifies the process to a single command, requiring only local ffmpeg and a MiMo API key, eliminating the need for GPUs or model downloads, and offering optional export to editing software like Jianying.

How It Works

The core approach involves a Claude Code plugin orchestrating five independent skills. It processes video via ASR, VLM, and scene analysis, leveraging a single MiMo API key for these functions. An Agent generates narration scripts, which are then synthesized into voiceovers using MiMo TTS. ffmpeg handles final assembly, audio mixing (including original audio ducking), and subtitle rendering. An optional "edit-mode cut" allows pre-editing the video before script generation for precise temporal alignment.

Quick Start & Requirements

Install the Claude Code plugin via the repository URL (https://github.com/worldwonderer/video-recap-skills). Requires ffmpeg (install via package manager: brew install ffmpeg on macOS, sudo apt install ffmpeg on Debian/Ubuntu, choco install ffmpeg on Windows) and Python 3.10+. Set the MiMo API Key as an environment variable (export MIMO_API_KEY=your-mimo-key). No GPU or model downloads are necessary. A demo video is available at https://github.com/user-attachments/assets/92698ec6-0d23-4f9f-8825-c3684ef57aff.

Highlighted Details

• Unified AI Backend: A single MiMo API key drives ASR, VLM, and TTS, simplifying setup.
• Intelligent Audio Ducking: Original audio is preserved but lowered during narration, with adjustable thresholds for seamless transitions.
• Pre-edit Workflow (--edit-mode cut): Enables users to first create a final video cut, then generate narration precisely aligned to that edited timeline.
• LLM Script Review: Incorporates an LLM-based quality check for generated narration scripts, mitigating hallucinations.
• Jianying Draft Export: Optionally generates a multi-track Jianying (CapCut) project file for further manual refinement.
• ffmpeg-centric Rendering: Core video assembly and rendering rely solely on ffmpeg, decoupling the process from installing editing software.

Maintenance & Community

The project references linux.do and external libraries (pyJianYingDraft, capcut-mate) for Jianying export structure. No specific community channels or contributor details are provided in the README.

Licensing & Compatibility

The project is licensed under the MIT License, permitting broad use, including commercial applications.

Limitations & Caveats

Relies heavily on the MiMo API for core AI functionalities, introducing external service dependency and potential costs. Advanced configuration requires consulting separate documentation. Testing necessitates specific scripts due to isolated skill modules. Jianying export references external libraries for draft structure.