AutoGLM-GUI provides a modern web-based graphical interface for automating Android device operations using AI. It targets developers and power users seeking a streamlined way to control multiple Android devices concurrently through conversational AI, offering real-time screen mirroring and direct touch interaction. The primary benefit is simplifying complex, repetitive mobile automation tasks by leveraging large language models (LLMs) and robust device integration.

How It Works

The system integrates deeply with Android Debug Bridge (ADB) for device control and utilizes scrcpy for low-latency H.264 video streaming, enabling real-time screen previews. Users interact via a chat interface, describing desired actions, which an AI agent interprets and executes on the Android device. It supports direct manipulation through the mirrored screen, translating clicks and gestures accurately to the device. The architecture features a backend managing independent PhoneAgent instances for each device and a frontend with a componentized design for clear state management and UI separation.

Quick Start & Requirements

• Installation: Install via pip: pip install autoglm-gui.
• Execution: Run using autoglm-gui --base-url <YOUR_OPENAI_COMPATIBLE_LLM_API_ENDPOINT>.
• Prerequisites: Python 3.10+, an Android device with USB debugging enabled, ADB installed and added to system PATH, and access to an OpenAI-compatible LLM API endpoint.
• Access: The web interface is accessible at http://localhost:8000 after startup.
• Docs: Source installation and development details are available within the repository.

Highlighted Details

• Multi-Device Control: Simultaneously manage and control multiple Android devices with isolated states, conversation histories, and video streams.
• Conversational AI Automation: Control devices through natural language prompts, with step-by-step execution visibility.
• Direct Touch Interaction: Perform clicks and gestures directly on the real-time screen preview, with accurate coordinate mapping and visual feedback.
• OpenAI Compatibility: Supports any LLM API endpoint compatible with OpenAI's API, including services like Zhipu BigModel and ModelScope, or self-hosted models via vLLM/SGLang.

Maintenance & Community

The project is built upon Open-AutoGLM by the zai-org team. Specific details regarding active maintenance, community channels (like Discord/Slack), or a public roadmap are not explicitly detailed in the provided README.

Licensing & Compatibility

AutoGLM-GUI itself is primarily licensed under the MIT license. However, it includes the ADB Keyboard APK (com.android.adbkeyboard), which is distributed under the GPL-2.0 license. This component is used as a standalone tool and is stated not to affect the MIT licensing of the main AutoGLM-GUI project. Compatibility for commercial use should consider the implications of the GPL-2.0 component.

Limitations & Caveats

The project requires specific setup including ADB and an LLM API endpoint. While it supports multiple devices, the architecture is designed for independent operation per device rather than complex inter-device coordination. The README does not explicitly list known bugs, alpha status, or unsupported platforms, suggesting a focus on core functionality.