AmbientGPT enables large language models (LLMs) like GPT-4o and local foundation models to access and utilize real-time information from your computer screen. This allows LLMs to understand your current context without explicit input, enhancing productivity for users who frequently switch between applications or need LLMs to reference on-screen content.

How It Works

AmbientGPT captures screen content and processes it to provide ambient knowledge to LLMs. This approach eliminates the need for manual context uploading, allowing LLMs to seamlessly integrate with the user's workflow by understanding the visual information presented on their screen.

Quick Start & Requirements

• Install dependencies: pip3 install -r requirements.txt and npm install && npm run dev.
• Requires an OpenAI API key added to test.py for GPT-4o.
• Local models require an ARM64 (M1, M2, M3) MacBook.
• MacOS app is pending Apple certification.

Highlighted Details

• Integrates GPT-4o and local foundation models.
• Provides ambient knowledge of the user's screen.
• Eliminates manual context uploading for LLMs.

Maintenance & Community

No information on maintainers, community channels, or roadmap is available in the provided README.

Licensing & Compatibility

The license is not specified in the README. Compatibility for commercial use or closed-source linking is not detailed.

Limitations & Caveats

Local model support is restricted to ARM64 MacBooks. GPT-4o functionality requires an OpenAI API key. A macOS application is planned but not yet available.