Mirror is a hackable, local-first AI-powered webcam application that provides real-time commentary on video feeds. It's designed for users interested in experimenting with multimodal AI, privacy-conscious individuals, and developers looking to repurpose AI for various applications. The primary benefit is its offline operation, ensuring privacy and free, continuous experimentation.

How It Works

Mirror leverages llama.cpp to run the Bakllava multimodal AI model locally. The application streams webcam video to the AI, which analyzes the frames and generates textual responses based on user-defined prompts. This approach allows for 100% local processing, eliminating internet dependency and ensuring data privacy.

Quick Start & Requirements

• Recommended Install: Pinokio (version 0.1.49+) for a one-click setup.
• Manual Install: Requires cloning the repository, llama.cpp, downloading specific Bakllava model files (ggml-model-q4_k.gguf, mmproj-model-f16.gguf), building llama.cpp (with CMake), installing Python dependencies (requirements.txt), and installing FFmpeg.
• Prerequisites: CMake, Visual Studio (Windows), FFmpeg, Python 3.x. GPU acceleration (-ngl 1) is supported and recommended for llama.cpp.
• Setup Time: Manual setup can be time-consuming due to multiple components and build steps.

Highlighted Details

• Real-time webcam analysis and response generation.
• 100% local and private operation, no internet required.
• Hackable via prompt modification or code tweaking.
• Utilizes llama.cpp for efficient LLM serving and Bakllava for multimodal capabilities.
• Web UI built with Gradio.

Maintenance & Community

• Project is maintained by cocktailpeanut.
• Credits acknowledge inspiration from Realtime Bakllava.

Licensing & Compatibility

• The repository's license is not explicitly stated in the README. llama.cpp is typically under MIT, and Gradio is Apache 2.0. Bakllava's license should be verified.

Limitations & Caveats

Manual installation is complex and involves orchestrating multiple backend services. The README strongly recommends the Pinokio installer due to the difficulty of manual setup.