Observer AI provides a platform for creating and running local, privacy-focused micro-agents that monitor user screens and react to events. It targets users who want to automate tasks or gain insights from their computer activity without sending data to external services. The benefit is localized, secure AI-driven automation.

How It Works

The system operates on a modular agent architecture, allowing users to define agents with specific "Sensors" (inputs like screen OCR, screenshots, clipboard, microphone), "Models" (LLMs run via Ollama), and "Tools" (functions for interaction). Agents are configured through a dashboard, with logic defined in either browser-based JavaScript or server-based Python, enabling diverse interaction capabilities from simple notifications to system-level commands.

Quick Start & Requirements

• Install/Run: Recommended via Docker Compose. Clone the repo and run docker-compose up --build.
• Prerequisites: Docker and Docker Compose installed.
• Access: Web UI at http://localhost:8080. Requires accepting a self-signed SSL certificate for https://localhost:3838.
• Docs: Observer App Link, Getting Started.

Highlighted Details

• Supports local inference via Ollama integration.
• Agents can be programmed in JavaScript (browser sandbox) or Python (Jupyter server for system access).
• Includes built-in sensors for screen OCR, screenshots, clipboard, and microphone.
• Provides tools for memory management, agent control, notifications, and communication (email, SMS, WhatsApp).

Maintenance & Community

• Project maintained by Roy Medina.
• Special thanks to the Ollama team.
• Links: GitHub Repository, Twitter, Website.

Licensing & Compatibility

• MIT License.
• Permissive license suitable for commercial use and integration with closed-source applications.

Limitations & Caveats

• Screen OCR and microphone transcription are currently limited to English only.
• WhatsApp messaging is temporarily restricted to static messages due to anti-spam measures.
• Python agents require a separate Jupyter server setup and configuration.