<2-3 sentences summarising what the project addresses and solves, the target audience, and the benefit.> EvalGPT is an LLM-powered code interpreter framework automating code generation, execution, and result delivery. It targets developers and power users seeking to enhance productivity through automated coding, efficient parallel task execution, and robust error handling, drawing architectural inspiration from Google's Borg system.

How It Works

EvalGPT employs a distributed master-agent architecture inspired by Google's Borg system. A central EvalGPT master node manages task planning, scheduling, and memory. It decomposes user requests into sub-tasks, assigning each to an EvalAgent. EvalAgents generate and execute code for their assigned sub-tasks, leveraging LLMs and potentially external tools. They communicate via shared memory managed by the master. Error handling involves task replanning by the master node to ensure robustness.

Quick Start & Requirements

• Primary Install: go install github.com/index-labs/evalgpt@latest
• Non-default prerequisites: Go, Python 3, OpenAI API Key.
• Configuration: Requires setting OPENAI_API_KEY environment variable and configuring the Python interpreter path (default /usr/bin/python3). Users must ensure necessary Python libraries are installed in the chosen interpreter.
• Demo: https://github.com/index-labs/evalgpt/assets/7857126/73417c1f-8866-47fb-951a-7fd03c9dbf41

Highlighted Details

• Automated code generation via LLMs (GPT-4, CodeLlama, Claude 2).
• Distributed master-agent architecture for parallel task execution.
• Robust error handling with dynamic task replanning.
• Scalable design for complex coding needs.
• Resource optimization inspired by Google Borg.
• Extensible runtime supporting external tool integration.

Maintenance & Community

The project is explicitly stated to be in "early stages of development" with active work ongoing. Users are encouraged to submit issues or PRs for questions or suggestions. No specific community links or contributor details are provided in the README.

Licensing & Compatibility

The README does not specify a license.

Limitations & Caveats

The project is in early development. Setup requires configuring an OpenAI API key and a Python interpreter, with users responsible for installing necessary Python libraries. The absence of a specified license may impact commercial adoption or integration.