Xiaomi Miloco is an open-source framework for advanced smart home automation, leveraging Xiaomi Home cameras for visual input and a self-developed Large Language Model (LLM) as its core intelligence. It aims to connect and control all IoT devices within a household through natural language commands and user-defined rules, offering a more creative and integrated smart home experience. The project targets users seeking enhanced smart home capabilities and developers interested in LLM-driven IoT solutions, with a key benefit being on-device processing for improved privacy and security.

How It Works

The system utilizes a novel approach by treating camera data streams as a primary source of perceptual information. A self-developed LLM analyzes these visual feeds to understand home scene events, enabling context-aware responses and actions. The architecture splits tasks into planning and visual understanding stages, employing an on-device LLM for video analysis to ensure family privacy. It seamlessly integrates with the Xiaomi Home ecosystem, allowing retrieval and execution of commands for Mi Home devices and scenes, and supports custom notifications.

Quick Start & Requirements

• Primary Install: One-click installation via bash script (wget -qO- .../install.sh) or by cloning the repository (git clone ... && bash scripts/install.sh). Docker is the recommended deployment method.
• Prerequisites:
  • Hardware: x64 CPU, NVIDIA 30 series or above GPU with minimum 8GB VRAM (12GB+ recommended), 16GB+ available storage for local models.
  • Software: Linux (Ubuntu 22.04+ recommended) or Windows (10+ with WSL2 support). macOS is not supported. Docker v20.10+, NVIDIA Driver with CUDA, NVIDIA Container Toolkit.
• Links: Docker Deployment Documentation, Usage Documentation, Contributing Guide.

Highlighted Details

• New Interaction Paradigm: Enables rule-setting and complex device control via natural language.
• Visual Data Integration: Leverages camera streams for LLM-driven scene analysis.
• On-Device LLM: Performs local video understanding for enhanced privacy and security.
• Ecosystem Support: Connects with Xiaomi Home devices, scenes, and notifications.
• Inference Backend: Utilizes the llama.cpp project for inference capabilities.

Maintenance & Community

The project acknowledges its core development team and thanks the llama.cpp open-source project. Community interaction is facilitated through GitHub Discussions. The team is actively hiring, with applications directed to xiaomi-miloco@xiaomi.com.

Licensing & Compatibility

This project is strictly limited to non-commercial use only. Use for developing applications, web services, or other software requires explicit written authorization from Xiaomi Corporation. The specific open-source license is detailed in the LICENSE.md file.

Limitations & Caveats

macOS is not currently supported. The most significant limitation is the strict non-commercial usage restriction, which may prevent adoption in many commercial or derivative software development contexts.