blurr  by Ayush0Chaudhary

AI agent for Android UI automation

Created 1 year ago
946 stars

Top 38.0% on SourcePulse

GitHubView on GitHub
Project Summary

Panda is an on-device AI agent for Android that automates complex, multi-step tasks by understanding natural language commands and interacting with the phone's UI. It is designed for users who want a more accessible and personalized mobile experience, acting as a proactive operator to handle tasks across various applications.

How It Works

Panda utilizes a multi-agent system architecture written in Kotlin. Its "Eyes & Hands" component, an Android Accessibility Service, enables it to perceive the screen's UI elements and execute touch gestures. The "Brain" is powered by LLM models for reasoning and planning, while the "Agent: Operator" acts as the executor, managing tasks with a "Notepad" for memory. This separation of concerns allows for complex and reliable operation.

Quick Start & Requirements

  • Installation: Clone the repository, set up API keys in local.properties (including sdk.dir and GEMINI_API_KEYS), and open the project in Android Studio.
  • Prerequisites: Android Studio, an Android device or emulator (API level 26+), and Gemini API keys.
  • Setup: Enable the "Panda" Accessibility Service within the app's settings upon first run.
  • Docs/Demos: Demos are available via provided GitHub asset links.

Highlighted Details

  • Proactive, on-device AI agent for Android.
  • Intelligent UI automation using Accessibility Service.
  • Persistent, personalized local memory for learned procedures and user preferences.
  • High-quality voice output via GCS Chirp.

Maintenance & Community

Contributions are welcome via issues or pull requests.

Licensing & Compatibility

Licensed under the MIT License, generally permissive for commercial use and closed-source linking.

Limitations & Caveats

The project is described as a "powerful proof-of-concept," and its roadmap indicates it is still under active development. The "What's Next" section notes that the roadmap list is not updated.

Health Check
Last Commit

6 months ago

Responsiveness

Inactive

Pull Requests (30d)
0
Issues (30d)
0
Star History
11 stars in the last 30 days

Explore Similar Projects

Feedback? Help us improve.