This repository curates resources for computer-use GUI agents, targeting researchers and developers in the AI agent space. It provides a structured collection of videos, blogs, papers, and projects related to automating computer operations and GUI interactions using large language models.

How It Works

The collection focuses on agents that interact with graphical user interfaces (GUIs) to perform tasks. It highlights research and projects leveraging multimodal large language models (MLLMs) for tasks like instruction-based computer control, GUI automation, and operator assistance. Key themes include grounding MLLMs in GUI environments, agent data collection, evaluation benchmarks, and safety considerations.

Quick Start & Requirements

This is a curated list of resources, not a runnable project. No installation or specific requirements are needed to browse the content.

Highlighted Details

• Extensive list of academic papers categorized by sub-topics like "OS Agents," "Web Agents," and "GUI Grounding."
• Includes links to practical demonstrations and setup guides for Anthropic's "Computer Use" capability.
• Features resources on agent evaluation benchmarks and safety considerations, such as "MobileSafetyBench" and "Adversarial Attacks on Multimodal Agents."
• Lists various open-source projects and frameworks related to computer-use agents, including OpenInterpreter and AgentStore.

Maintenance & Community

The repository is under continuous construction, with an open invitation for contributions and feedback. Specific contributors or community channels are not detailed in the README.

Licensing & Compatibility

The repository itself is not a software project with a license. The linked resources may have various licenses; users should verify individual project licenses.

Limitations & Caveats

As a curated list, the repository's content is dependent on the availability and maintenance of the linked external resources. The "under construction" status implies the collection may be incomplete or subject to change.