This project enables an AI agent to securely interact with a Linux desktop environment, controlled by open-source LLMs. It's designed for developers and researchers looking to build AI-powered automation tools that can perform complex tasks on a computer, offering a secure sandbox and flexible LLM integration.

How It Works

The system leverages the E2B Desktop Sandbox to provide a secure, isolated Linux environment. The AI agent controls the sandbox via keyboard, mouse, and shell commands, with its actions streamed live to the client. This approach allows for complex, multi-step operations without compromising the host system, and its modular design facilitates easy integration of various LLMs for vision, action, and grounding.

Quick Start & Requirements

• Install: poetry install followed by poetry run start
• Prerequisites: Python 3.10+, Git, ffmpeg, E2B API key, and API keys for chosen LLM providers (e.g., Groq, OpenAI, Anthropic). A Hugging Face token is required to bypass rate limits.
• Setup: Requires cloning the repository, setting environment variables in a .env file, and installing dependencies via Poetry.
• Docs: E2B Desktop Sandbox

Highlighted Details

• Supports over 10 LLM providers including Groq, OpenAI, Anthropic, Gemini, and open-source models via Hugging Face Spaces (OS-Atlas, ShowUI).
• Allows pausing the agent and providing new prompts at any time during execution.
• Designed for Ubuntu but adaptable to other operating systems.
• Live display streaming of the sandbox environment to the client.

Maintenance & Community

The project is maintained by e2b-dev. Further community engagement details are not explicitly provided in the README.

Licensing & Compatibility

The repository's license is not explicitly stated in the README. Compatibility for commercial use or closed-source linking would require clarification on the licensing terms.

Limitations & Caveats

The project requires specific API keys for E2B and LLM providers, which may incur costs. The setup involves managing environment variables and dependencies via Poetry. The README does not detail specific performance benchmarks or known limitations of the sandbox environment.