This project provides an all-in-one demonstration for LLaVA, enabling interactive image chat, segmentation, and generation/editing. It targets researchers and users interested in multimodal AI capabilities, offering a unified interface for complex visual tasks.

How It Works

LLaVA-Interactive integrates multiple state-of-the-art models, including LLaVA for vision-language understanding, SEEM for comprehensive segmentation, and GLIGEN for grounded text-to-image generation. This combination allows for a seamless workflow where users can converse with images, precisely segment objects, and generate or edit images based on textual prompts.

Quick Start & Requirements

• Install via conda and pip.
• Requires CUDA 11.7 or above.
• Recommended for desktop computers.
• Project Page
• Demo (Note: Live demo disabled as of June 10, 2024)
• Paper

Highlighted Details

• Unified interface for image chat, segmentation, and generation/editing.
• Integrates LLaVA, SEEM, and GLIGEN models.
• Utilizes LaMa for background hole filling.

Maintenance & Community

• The project is associated with LLaVA-VL.
• Related projects include LLaVA, SEEM, and GLIGEN.

Licensing & Compatibility

• Licensed under the Apache License.
• GLIGEN is MIT licensed.
• Intended for non-commercial use only.
• Subject to LLaMA model license, OpenAI Terms of Use, and ShareGPT Privacy Practices.

Limitations & Caveats

The live demo website is currently disabled. The service is a research preview with limited safety measures and may generate offensive content. It is not intended for commercial use.