This repository serves as a comprehensive, curated list of resources for AI agents designed for computer use (ACU). It targets researchers, developers, and power users interested in autonomous programs that can interact with digital interfaces via clicks, keystrokes, and API calls to achieve user-defined goals. The benefit is a centralized, up-to-date knowledge base on this rapidly evolving field.

How It Works

The repository is structured into categorized lists of research papers, frameworks, models, datasets, benchmarks, and tools related to ACU. It highlights recent advancements, novel approaches, and key projects, providing links to official resources, code repositories, and preprints. The curation focuses on agents that combine perception, decision-making, and control capabilities for interacting with computer and mobile device GUIs.

Quick Start & Requirements

This repository is a curated list and does not require installation or execution. Users can browse the resources directly.

Highlighted Details

• Extensive coverage of recent research papers (late 2023 - early 2025) on ACU, including surveys and specific framework advancements.
• Categorized lists of open-source and commercial frameworks/models like AutoGen, Auto-GPT, LaVague, OpenInterpreter, and commercial offerings from Anthropic and Multion.
• Detailed sections on UI grounding techniques, datasets for training and evaluation, and benchmarks for assessing agent performance across various platforms (web, Android, Windows).
• Includes a section on safety considerations and potential attack vectors for ACU.

Maintenance & Community

The project is actively maintained, with frequent additions of new papers and projects. A link to an X Community is provided for discussion and contributions.

Licensing & Compatibility

The repository itself is a list of links and does not have a specific license. The linked resources will have their own licenses, which users must adhere to.

Limitations & Caveats

As a curated list, the repository's content is dependent on the availability and discoverability of public resources. The rapid pace of development means some information may become outdated quickly, requiring continuous updates.