imaginAIry is a Python library for generating AI images and videos, targeting developers and power users who want programmatic control over diffusion models. It simplifies complex AI art generation workflows, offering features like image-to-image, outpainting, ControlNet integration, and video generation from prompts or existing images.

How It Works

The library leverages the refiners library for its core diffusion model operations, enabling support for cutting-edge features like SDXL and image prompts. It provides a Pythonic API and a command-line interface (CLI) for flexible usage. Key architectural choices include automatic model weight downloading, prompt expansion with phrase lists, and integration with various control mechanisms like ControlNet and image editing instructions.

Quick Start & Requirements

• Install via pip: pip install imaginairy
• Requirements: Python 3.10 (3.11 not supported), ~10GB disk space for models, CUDA-enabled GPU with >=11GB VRAM (or M1 processor). For macOS, Rust and setuptools-rust are required.
• Official Docs: https://brycedrennan.github.io/imaginAIry/

Highlighted Details

• Supports Stable Video Diffusion for AI video generation.
• Integrates Spandrel for various upscaling models.
• Offers extensive ControlNet support (Openpose, Canny, HED, Depth, Normal, Shuffle) for guided image generation.
• Advanced prompt engineering features including weighted prompts, prompt expansion, and prompt-based masking.
• Includes image editing capabilities via InstructPix2Pix and prompt-based modifications.

Maintenance & Community

The project is actively maintained with frequent updates, including recent additions like SDXL support, image prompts, and frame interpolation for smoother video. Links to community resources are not explicitly provided in the README.

Licensing & Compatibility

The project appears to be available under a permissive license, though specific details are not explicitly stated in the README. Compatibility for commercial use or closed-source linking would require further investigation into the licensing of its dependencies.

Limitations & Caveats

SDXL models do not yet support inpainting or ControlNets. InstructPix2Pix functionality was noted as broken as of version 14.0.0. Windows support is mentioned as "sometimes" working, with specific instructions for PyTorch installation.