Automate your mobile devices with natural language commands using Mobilerun Portal, an Android accessibility service. It provides real-time UI element feedback and data collection, targeting UI testers, automation developers, and accessibility assessors. This offers a flexible toolkit for deep mobile control, interaction automation, and integration with external systems.

How It Works

The core approach utilizes the Android Accessibility Service API to create an interactive overlay that visually highlights UI elements like clickable, editable, and focusable components. It exposes multiple local control interfaces, including HTTP, WebSocket (JSON-RPC), and a ContentProvider accessible via ADB, facilitating integration with various automation frameworks. A key differentiator is the reverse WebSocket connection, enabling cloud control and real-time screen/event streaming.

Quick Start & Requirements

• Primary install / run command (pip, Docker, binary, etc.). Install the Android application (.apk).
• Non-default prerequisites and dependencies (GPU, CUDA >= 12, Python 3.12, large dataset, API keys, OS, hardware, etc.). Android device (API level 30+). Enable the "Mobilerun Portal" accessibility service in Android Settings and grant overlay permission when prompted.
• Estimated setup time or resource footprint. Setup involves app installation and service enablement, typically a few minutes.
• If they are present, include links to official quick-start, docs, demo, or other relevant pages. Referenced documentation sections: Local API, Triggers and Events, Reverse Connection.

Highlighted Details

• Real-time interactive overlay highlighting UI elements (clickable, checkable, editable, scrollable, focusable).
• Multiple local control APIs: HTTP socket server (default port 8080), WebSocket server (default port 8081), and ADB-accessible ContentProvider.
• Reverse WebSocket connection for cloud control and WebRTC screen streaming with auto-accept support.
• In-app and remote APK installation (including split APKs) with optional auto-accept.
• Notification event streaming over WebSocket with per-event toggles.
• Extensive ADB commands via ContentProvider for querying device state (a11y tree, phone state, installed apps) and executing actions (keyboard input, key events, configuration changes).

Maintenance & Community

• Continuous Integration: Utilizes GitHub Actions for automated building and releasing upon code changes.
• Community: No specific community channels (e.g., Discord, Slack) or contribution guidelines were detailed in the provided README.

Licensing & Compatibility

• License Type: Not specified in the provided README.
• Compatibility Notes: Requires Android API level 30 (Android 11.0) or higher.

Limitations & Caveats

The project requires a minimum Android API level of 30 (Android 11.0). Enabling the accessibility service and overlay permissions may introduce security considerations and vary in behavior across different Android versions and device manufacturers. Crucially, the project's licensing terms are not disclosed in the provided documentation, hindering assessment for commercial or closed-source integration.