OpenGUI is an Android GUI agent framework enabling AI to perceive, plan, and operate real mobile applications. It targets engineers, researchers, and power users seeking to automate mobile tasks and leverage AI for phone interaction, offering a robust platform for complex, long-running mobile workflows.

How It Works

OpenGUI functions as a mobile operator system with explicit orchestration layers. It employs a main graph agent for high-level planning and an executor subgraph for device-side operations. The system handles screen understanding, task planning, action execution, review, and recovery, processing UI elements, planning steps, executing mobile actions, and returning structured results. Its architecture supports splitting planning and vision-language model (VLM) execution across different providers, and it features a device path with standby and execution sockets for persistent connections.

Quick Start & Requirements

The quickest setup involves using AI models like Claude Code or Codex with the skills/open-gui-bootstrap/SKILL.md skill. Alternatively, manual setup involves running server and client scripts:

cd server
./start.sh
cd client
./start.sh

After setup, initiate tasks via the CLI, e.g., pnpm opengui -- do "Observe the current Android screen and summarize what you see" --json.

Prerequisites:

• Android phone or emulator
• USB debugging enabled
• AccessibilityService enabled
• Model API keys
• PostgreSQL and Redis (for backend)
• Node.js/pnpm (for server CLI)

Links:

• Manual Setup Guide: docs/get-started.md
• Bootstrap Skill: skills/open-gui-bootstrap/SKILL.md
• Discord Community: docs/DISCORD.md

Highlighted Details

• Designed for long-running tasks, incorporating progress tracking, review, and recovery mechanisms.
• Features a "plan before action, summarize after execution" loop for structured task completion.
• Supports remote operation via Feishu, Telegram, Discord, or REST API through a standby dispatch path.
• Model routing allows flexible assignment of planning and VLM tasks to different AI providers.

Maintenance & Community

Recent updates include remote control via REST API and CLI, enhanced Discord integration for task dispatch, and hardened backend startup procedures. A Discord community channel is available for support and remote task dispatch.

Licensing & Compatibility

OpenGUI is released under the Business Source License 1.1 (BUSL-1.1). Non-production use, modification, distribution, and copying are permitted. Production, commercial use, hosted services, or integration into commercial products require a separate commercial license from Core-Mate. The license transitions to Apache License 2.0 on 2030-04-29, at which point it becomes OSI-approved open source.

Limitations & Caveats

Operation requires an Android device/emulator with specific permissions (USB debugging, AccessibilityService). Execution quality is contingent on the AI model, app UI, network conditions, and task complexity. It is not an always-on OS assistant, with tasks initiated manually or via configured channels. Reliability for long-running tasks requires further real-world validation, and more ready-to-run examples and benchmarks are needed.