MotionAgent is an AI-powered tool for generating videos from user-provided scripts, targeting creators and developers who want to automate video production. It offers a comprehensive workflow from scriptwriting to final video and music generation, leveraging large language models and diffusion models.

How It Works

MotionAgent integrates multiple AI models to achieve its functionality. It uses LLMs like Qwen-7B-Chat for script generation based on themes, diffusion models (SDXL 1.0) for generating movie stills from scripts, image-to-video models (I2VGen-XL) for video creation, and MusicGen for background music composition. This modular approach allows for a complete pipeline from text to multimedia output.

Quick Start & Requirements

• Install: Clone the repository and install dependencies via pip3 install -r requirements.txt. Run the application with python3 app.py.
• Prerequisites: Python 3.8, PyTorch 2.0.1, CUDA 11.7, Ubuntu 20.04.
• Hardware: Requires an Nvidia A100 40GB GPU (minimum 36GB VRAM) and over 50GB disk space.
• Docs: ModelScope library

Highlighted Details

• Script generation using LLMs (e.g., Qwen-7B-Chat).
• Movie still generation via SDXL 1.0.
• Video generation from images using I2VGen-XL.
• Custom background music generation with MusicGen.

Maintenance & Community

The project is powered by the ModelScope community, an open-source model ecosystem from Damo Academy. Further details on contributing models are available.

Licensing & Compatibility

Licensed under the Apache License (Version 2.0). This license is permissive and generally compatible with commercial use and closed-source linking.

Limitations & Caveats

The current setup is verified for a specific environment (Python 3.8, CUDA 11.7, Ubuntu 20.04) and requires significant GPU memory (36GB+). The --clear_cache flag is noted for low-disk environments, implying potential re-downloading of large models with speed impacts.