This repository serves as a curated list of research papers focused on GUI agents, encompassing datasets, benchmarks, agent frameworks, and foundation models specifically designed for or applied to GUI interactions. It aims to provide a comprehensive resource for researchers and practitioners in the field of AI agents that interact with graphical user interfaces across web, mobile, and desktop environments.

How It Works

The repository organizes papers by environment (Web, Mobile, Desktop, GUI, Misc) and keywords (framework, benchmark, dataset, model, etc.). Each entry includes authors, institutions, publication details, and a concise TLDR summary, allowing for quick assessment of relevance and contribution. The list is actively updated with recent publications, reflecting the rapid advancements in GUI agent research.

Quick Start & Requirements

• Installation: No installation required; this is a curated list.
• Requirements: Access to the internet to view linked papers.
• Resources: Primarily requires time for reading and research.
• Links: The README provides direct links to papers on arXiv, GitHub, and other publication venues.

Highlighted Details

• Comprehensive coverage of GUI agents across web, mobile, and desktop platforms.
• Categorization by environment and keywords for efficient filtering.
• Active curation with recent publications in the field.
• Includes papers on foundational models, agent frameworks, datasets, and benchmarks.

Maintenance & Community

• Maintained by the OSU-NLP-Group.
• The list is updated regularly, reflecting ongoing research.

Licensing & Compatibility

• The repository itself is likely under a permissive license (e.g., MIT, Apache 2.0), but the licensing of the linked papers depends on their respective publishers.

Limitations & Caveats

• This is a list of papers, not a software library; it does not provide code or runnable agents.
• The quality and depth of summaries are dependent on the original paper's abstract and TLDR.