AutoGLM For Android addresses the need for on-device mobile automation by providing a native Android application that replaces PC-based solutions. It allows users to control their Android devices using natural language commands, making complex automation accessible to a broader audience without requiring technical expertise or external hardware. The app streamlines task execution, from app launching to message sending, directly from the user's phone.

How It Works

This project re-architects the Open-AutoGLM framework into a standalone Android application. It bypasses the traditional PC-ADB connection by leveraging the Shizuku service to obtain necessary system permissions for interacting with the device. Users input natural language descriptions of tasks, which are then processed by an AI model. The model plans and executes a sequence of actions, such as taps, swipes, and text inputs, with progress visualized in a floating window. The system supports various OpenAI-compatible models, including those with image understanding capabilities, for interpreting screen content and generating commands.

Quick Start & Requirements

• Installation: Download and install the latest APK from the project's Releases page.
• Prerequisites:
  • Android 7.0 (API 24) or higher.
  • Shizuku app installed and activated (Wireless Debugging recommended, ADB or Root also supported).
  • Network connectivity to an AI model API service.
  • Required permissions: Shizuku, Floating Window, Network, Keyboard.
• Model Configuration: Requires configuring an AI model API endpoint (Base URL, Model Name, API Key). Examples include Zhipu BigModel and ModelScope.
• Setup: Requires installing the app, activating Shizuku, granting permissions, and configuring the AI model.

Highlighted Details

• PC-Free Operation: Fully functional on the Android device itself.
• Natural Language Interface: Tasks are described using plain language.
• Shizuku Integration: Enables system-level operations like screen taps and swipes.
• Floating Window: Provides real-time feedback on task execution steps and AI reasoning.
• Multi-Model Compatibility: Supports any model API compatible with OpenAI's format and capable of image understanding.
• Native Android Experience: Features a Material Design interface for a smooth user experience.

Maintenance & Community

• Contact: Primarily through GitHub Issues and email (luokavin@foxmail.com).
• Dependencies: Relies on Shizuku for permissions and external AI model APIs.

Licensing & Compatibility

• License: MIT License.
• Compatibility: The MIT license permits commercial use and integration into closed-source projects.

Limitations & Caveats

• Shizuku Dependency: The application's core functionality is contingent on a properly configured and running Shizuku service.
• External AI Model: Requires access to and configuration of an external AI model API, which may involve costs or specific setup.
• Sensitive Page Restrictions: Screen capture and interaction are automatically disabled on sensitive pages (e.g., payment, password entry) due to system security.
• Permission Requirements: Necessitates granting several sensitive system permissions.