This project provides a Home Assistant integration for analyzing images and video streams using multimodal Large Language Models (LLMs). It targets Home Assistant users who want to leverage AI for intelligent analysis of camera feeds, video files, and events, enabling features like object recognition, event summarization, and timeline tracking.

How It Works

LLM Vision integrates with various LLM providers, including OpenAI, Anthropic Claude, Google Gemini, and local solutions like Ollama and LocalAI. It processes visual data (images, video, live feeds) and uses LLMs to extract information, identify objects, people, or pets, and maintain a chronological timeline of events. This allows for intelligent sensor updates and natural language querying of past events.

Quick Start & Requirements

• Install via Home Assistant Community Store (HACS).
• Requires Home Assistant setup.
• Provider API keys or local LLM setup (e.g., Ollama) are necessary.
• Detailed setup instructions for providers are available at: https://llm-vision.gitbook.io/getting-started/setup/providers

Highlighted Details

• Supports a wide range of LLM providers, including OpenAI-compatible endpoints.
• Analyzes images, video files, live camera feeds, and Frigate events.
• Maintains a timeline of analyzed events, viewable on a dashboard.
• Can intelligently summarize camera event notifications via a blueprint.

Maintenance & Community

• Project is actively maintained.
• Community discussions are available via the Home Assistant Community forum.

Licensing & Compatibility

• The repository does not explicitly state a license in the provided README.

Limitations & Caveats

• The specific license is not mentioned, which may impact commercial use or integration into closed-source projects.