EVAL is an AI agent designed to execute user requests by autonomously searching, coding, running, and testing on the internet. It targets users who need to automate complex tasks, from generating web applications to performing multimodal data analysis, by leveraging a flexible, self-evolving toolset.

How It Works

EVAL integrates LangChain, LlamaIndex, and Visual ChatGPT to create a versatile agent. It employs a modular architecture allowing it to dynamically create, modify, and execute code to build new tools as needed. This approach enables it to handle multimodal inputs (text, image, dataframe, with audio/video planned) and serve blocking processes like web applications.

Quick Start & Requirements

• Install/Run: Use docker-compose up --build eval (CPU) or docker-compose up --build eval.gpu (GPU).
• Prerequisites: Requires an OpenAI API key (OPENAI_API_KEY). GPU version has significant dependencies and is noted as unstable.
• Setup: Docker installation is required.
• Docs: Web GUI available at http://localhost:8000. API documentation via POST /api/execute.

Highlighted Details

• Supports multimodal conversation including image understanding (BLIP) and generation (Stable Diffusion).
• Can create and manage its own tools through code writing, patching, and deletion.
• Includes built-in tools for terminal access, file operations, web search (Google, Bing, Wikipedia), and custom database lookups.
• Offers both synchronous and asynchronous API execution endpoints.

Maintenance & Community

The project references LangChain, LlamaIndex, and Visual ChatGPT. No specific community channels or roadmap are detailed in the README.

Licensing & Compatibility

The README does not explicitly state a license. It references other projects with various licenses, implying potential compatibility considerations for commercial use.

Limitations & Caveats

The GPU version is described as "much heavier and unstable for now." Several features are marked as TODO, including GUI, memory saving, session management, and expanded multimodal support (audio, video).