cua  by trycua

AI agent framework for computer OS control in virtual containers

created 6 months ago
9,088 stars

Top 5.7% on sourcepulse

GitHubView on GitHub
Project Summary

c/ua (Computer-Use Agent) is a framework designed to enable AI agents to control full operating systems within high-performance virtual containers. It targets developers and researchers building AI agents that require interaction with desktop environments, offering near-native performance on Apple Silicon and flexible OS support.

How It Works

c/ua leverages Apple's Virtualization.framework for high-performance macOS and Linux virtualization on Apple Silicon, achieving up to 97% of native speed. It provides a Computer-Use Interface (CUI) that allows AI systems to observe and control these virtual environments, enabling interaction with applications, web browsing, and complex workflows. The framework integrates with various LLM providers for agentic capabilities.

Quick Start & Requirements

  • Install Lume CLI:
    /bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/trycua/cua/main/libs/lume/scripts/install.sh)"
    
  • Pull macOS CUA image:
    lume pull macos-sequoia-cua:latest
    
  • Install Python libraries:
    pip install cua-computer cua-agent[all]
    
  • Prerequisites: Mac with Apple Silicon (M1/M2/M3/M4), macOS 15 (Sequoia) or newer, Python 3.10+.
  • Resources: 30GB+ disk space recommended.
  • Docs: Computer, Agent, Notebooks

Highlighted Details

  • Near-native performance (up to 97%) on Apple Silicon via Virtualization.framework.
  • Supports both macOS and Linux virtual environments within a single framework.
  • Integrated Computer-Use Interface for AI agent interaction with OS.
  • Built-in LLM integration for OpenAI, Anthropic, and other providers.
  • Gradio UI available for interactive agent deployment.

Maintenance & Community

The project has a Discord community (https://discord.com/invite/mVnXXpdE85) and lists several active contributors.

Licensing & Compatibility

Licensed under the MIT License. Microsoft's OmniParser, used within the project, is licensed under CC-BY-4.0. Compatible with commercial use under MIT terms.

Limitations & Caveats

Requires macOS 15 (Sequoia) or newer, which is currently in beta. The project is actively developed, and specific features or stability may vary.

Health Check
Last commit

2 days ago

Responsiveness

1 week

Pull Requests (30d)
7
Issues (30d)
5
Star History
3,721 stars in the last 90 days

Explore Similar Projects

Starred by Chip Huyen Chip Huyen(Author of AI Engineering, Designing Machine Learning Systems) and Robert Stojnic Robert Stojnic(Creator of Papers with Code).

Agent-S by simular-ai

1.2%
6k
Agentic framework for autonomous computer interaction
created 9 months ago
updated 20 hours ago
Starred by Elie Bursztein Elie Bursztein(Cybersecurity Lead at Google DeepMind), Chip Huyen Chip Huyen(Author of AI Engineering, Designing Machine Learning Systems), and
1 more.

SuperAGI by TransformerOptimus

0.2%
17k
Open-source framework for autonomous AI agent development
created 2 years ago
updated 6 months ago
Feedback? Help us improve.