AI-Driven Scientific and Algorithmic Discovery

SkyDiscover is a flexible, modular framework for AI-driven scientific and algorithmic discovery, offering a unified interface for implementing, running, and benchmarking over 200 optimization tasks. It introduces novel adaptive algorithms, AdaEvolve and EvoX, which leverage LLMs to dynamically adjust or evolve optimization strategies, aiming to accelerate discovery and provide fairer comparisons against existing state-of-the-art methods.

How It Works

The framework employs a modular design, supporting integration and comparison of various discovery algorithms. Its core innovations are AdaEvolve, which adaptively modifies optimization behavior based on progress, and EvoX, which uses LLMs to dynamically evolve the optimization strategy itself. SkyDiscover also provides native implementations and supports external backends for algorithms like OpenEvolve, GEPA, and ShinkaEvolve, facilitating direct benchmarking. Solutions are generated and refined iteratively by LLMs, guided by user-defined evaluators.

Quick Start & Requirements

Installation uses uv: uv sync for base, uv sync --extra <benchmark_extras> (e.g., math, external) for specific dependencies. Python 3.10+ is required, along with LLM API keys (e.g., OPENAI_API_KEY). The project supports various LLM models via LiteLLM compatibility. Links to benchmark-specific setup and configuration templates are available.

Highlighted Details

• Achieves state-of-the-art open-source results across ~200 optimization benchmarks, with AdaEvolve and EvoX matching or exceeding AlphaEvolve and human SOTA.
• Demonstrates significant improvements: ~34% median score increase on Frontier-CS (172 problems) and strong performance on math/systems tasks.
• Shows real-world impact, including 41% lower cross-cloud transfer cost and 14% better GPU load balance.
• Features an integrated live monitor for visualizing progress, code diffs, metrics, and AI summaries, with options for human feedback.

Maintenance & Community

The project is marked "under active development." Contact information for key researchers is provided (lshu@berkeley.edu, mert_cemri@berkeley.edu, shubham3@berkeley.edu). No specific community channels or detailed roadmap are listed.

Licensing & Compatibility

The README does not specify a software license. This omission requires clarification for adoption decisions, particularly regarding commercial use or integration with closed-source projects.

Limitations & Caveats

The project is under active development, implying potential for breaking changes. Integration with certain external algorithms requires manual installation. A critical limitation for adoption is the absence of a stated software license.