This project provides a fully local, AI-driven workflow for automatically generating video content from novel text. It targets users aiming to create videos from literature with minimal effort, offering an end-to-end solution that integrates TTS, image generation, and direct export to Jianying for simplified post-production.

How It Works

The system orchestrates multiple AI services via the Model Context Protocol (MCP) using a Go backend. It processes novel text through automatic chapter splitting, then leverages Ollama for content analysis and prompt optimization. IndexTTS2 generates high-quality audio (with voice cloning support), while DrawThings creates visuals based on chapter content or lyrics. Aegisub handles subtitle generation, culminating in a Jianying-compatible project structure for easy editing.

Quick Start & Requirements

• Primary Install: go run main.go (starts MCP and Web UI).
• Prerequisites: macOS (tested), Go 1.25+, 16GB+ RAM (32GB recommended), Apple Silicon GPU (Metal), 100GB+ storage. Requires Ollama (with qwen3:4b model), DrawThings (HTTP on port 7861), and IndexTTS2 (running on http://localhost:7860). FFmpeg and Aegisub are also needed.
• Setup: Input novel text into input/NovelName/NovelName.txt. Optional reference audio in assets/ref_audio/.
• Web UI: Access at http://localhost:8080.
• Jianying Client: Version 3.4.1 recommended.
• Links: Online Demo: https://yadou.net. Docs: SYSTEM_ARCHITECTURE.md, USER_GUIDE.md.

Highlighted Details

• Image MV Generation: Creates music video-style images from lyrics using AI prompts.
• MCP Integration: Extensible via the Model Context Protocol, enabling integration with AI agents like Ollama Desktop.
• Direct Jianying Export: Generates .json project files directly importable into Jianying, streamlining the editing process.
• Local Processing: Ensures data privacy and offline functionality.

Maintenance & Community

• Contact: Email yuanhaozhuzhu@hotmail.com, QQ Group 1033223644.
• Contributions: Issues and Pull Requests are welcomed.

Licensing & Compatibility

• License: MIT License.
• Compatibility: Permissive MIT license allows for commercial use and integration into closed-source projects.

Limitations & Caveats

The system is primarily tested on macOS and explicitly recommends Jianying client version 3.4.1, suggesting potential compatibility issues with other OS or versions. It has substantial hardware requirements (RAM, storage, specific GPU) and relies on correctly configured, independently running AI services (Ollama, DrawThings, IndexTTS2) on specific ports, increasing setup complexity.