<AI agents can now interact with real Android and iOS applications through natural language commands, enabling automated task execution and data extraction on mobile devices. This open-source project is targeted at developers and power users seeking to automate mobile workflows, with the primary benefit of leveraging AI for intuitive and powerful mobile device control.>

How It Works

Mobile-use employs AI agents to interpret natural language instructions and translate them into actions on a mobile device. It intelligently navigates application UIs, allowing for complex task automation and data scraping. The system is designed to be extensible, supporting various Large Language Models (LLMs) for agent customization and offering a flexible configuration for different AI providers.

Quick Start & Requirements

• Installation: Docker is recommended for a quick start, with specific scripts provided for Linux/macOS (mobile-use.sh) and Windows (mobile-use.ps1). Manual setup involves cloning the repository, setting environment variables (including an OpenAI API key), and installing dependencies using uv.
• Prerequisites: Android Debug Bridge (ADB) for Android devices, Xcode for iOS development (macOS only), Docker, Python (via uv), and the Maestro framework are required. Physical Android devices need USB debugging enabled; Android emulators can also be used. iOS simulators are supported on macOS.
• Setup: Requires API keys and potentially configuring LLM settings. Docker setup time is minimal if Docker is already installed.
• Links: Discord

Highlighted Details

• Achieved #1 open-source pass@1 on the AndroidWorld benchmark.
• Supports data scraping with natural language descriptions for structured output (e.g., JSON).
• Extensible LLM configuration allows integration with various AI models.

Maintenance & Community

The project actively encourages contributions and community involvement, with a Discord server available for discussion and support.

Licensing & Compatibility

• License: MIT License.
• Compatibility: Compatible with physical Android phones (via USB), Android emulators, and iOS simulators on macOS. Physical iOS devices are not yet supported.

Limitations & Caveats

The Docker quickstart is currently limited to Android devices/emulators. Troubleshooting network or Docker image authorization issues may be required. Physical iOS device support is a stated future enhancement.