cua  by trycua

AI agent framework for computer OS control in virtual containers

Created 1 year ago
20,441 stars

Top 2.6% on SourcePulse

GitHubView on GitHub
Project Summary

c/ua (Computer-Use Agent) is a framework designed to enable AI agents to control full operating systems within high-performance virtual containers. It targets developers and researchers building AI agents that require interaction with desktop environments, offering near-native performance on Apple Silicon and flexible OS support.

How It Works

c/ua leverages Apple's Virtualization.framework for high-performance macOS and Linux virtualization on Apple Silicon, achieving up to 97% of native speed. It provides a Computer-Use Interface (CUI) that allows AI systems to observe and control these virtual environments, enabling interaction with applications, web browsing, and complex workflows. The framework integrates with various LLM providers for agentic capabilities.

Quick Start & Requirements

  • Install Lume CLI:
    /bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/trycua/cua/main/libs/lume/scripts/install.sh)"
    
  • Pull macOS CUA image:
    lume pull macos-sequoia-cua:latest
    
  • Install Python libraries:
    pip install cua-computer cua-agent[all]
    
  • Prerequisites: Mac with Apple Silicon (M1/M2/M3/M4), macOS 15 (Sequoia) or newer, Python 3.10+.
  • Resources: 30GB+ disk space recommended.
  • Docs: Computer, Agent, Notebooks

Highlighted Details

  • Near-native performance (up to 97%) on Apple Silicon via Virtualization.framework.
  • Supports both macOS and Linux virtual environments within a single framework.
  • Integrated Computer-Use Interface for AI agent interaction with OS.
  • Built-in LLM integration for OpenAI, Anthropic, and other providers.
  • Gradio UI available for interactive agent deployment.

Maintenance & Community

The project has a Discord community (https://discord.com/invite/mVnXXpdE85) and lists several active contributors.

Licensing & Compatibility

Licensed under the MIT License. Microsoft's OmniParser, used within the project, is licensed under CC-BY-4.0. Compatible with commercial use under MIT terms.

Limitations & Caveats

Requires macOS 15 (Sequoia) or newer, which is currently in beta. The project is actively developed, and specific features or stability may vary.

Health Check
Last Commit

16 hours ago

Responsiveness

1 day

Pull Requests (30d)
7
Issues (30d)
2
Star History
18 stars in the last 30 days

Explore Similar Projects

Starred by Junyang Lin Junyang Lin(Core Maintainer at Alibaba Qwen), Eric Zhu Eric Zhu(Coauthor of AutoGen; Research Scientist at Microsoft Research), and
2 more.

OpenSandbox by opensandbox-group

0.1%
12k
Sandbox platform for AI and LLM applications
Created 8 months ago
Updated 1 week ago
Starred by Dan Abramov Dan Abramov(Core Contributor to React; Coauthor of Redux, Create React App), Carol Willing Carol Willing(Core Contributor to CPython, Jupyter), and
34 more.

pi by earendil-works

0.0%
75k
AI agent framework and LLM deployment tools
Created 1 year ago
Updated 1 day ago
Feedback? Help us improve.