This repository provides ASI-Arch, a highly autonomous, multi-agent framework for discovering novel scientific model architectures, specifically focusing on linear attention mechanisms. It's designed for researchers and engineers seeking to automate and accelerate the process of hypothesis generation, implementation, and empirical validation of new model designs, having already discovered 106 state-of-the-art linear attention architectures.

How It Works

ASI-Arch employs a multi-agent system orchestrated by a central pipeline. This pipeline autonomously hypothesizes new architectural concepts, implements them as code, and empirically validates their performance through systematic experimentation. It leverages a MongoDB-based Architecture Database for storing historical data and enabling agent information retrieval, and a MongoDB-powered Cognition Base for providing relevant research insights via a RAG approach to guide the discovery process.

Quick Start & Requirements

• Installation: Clone the repository, create a Conda environment (conda create -n asi-arch python=3.10), activate it (conda activate asi-arch), and install dependencies (pip install -r requirements.txt, PyTorch with CUDA, and component-specific requirements).
• Prerequisites: Python 3.8+, MongoDB 4.4+, Docker & Docker Compose, CUDA-compatible GPU (recommended), minimum 16GB RAM (32GB recommended).
• Setup: Requires starting MongoDB and RAG API services via Docker Compose.
• Running Discovery: Execute python pipeline/pipeline.py within the activated Conda environment.
• Documentation: Paper (Coming Soon)

Highlighted Details

• Discovered 106 novel linear attention architectures achieving state-of-the-art performance.
• Features an autonomous loop: hypothesize, implement, validate, analyze, and update.
• Utilizes FAISS for high-speed vector similarity search to ensure novelty.
• Integrates a RAG service for grounding agent decisions in scientific literature.

Maintenance & Community

The project is inspired by FLAME, LM-Evaluation-Harness, and Flash Linear Attention (FLA). Citation details are provided in BibTeX format.

Licensing & Compatibility

The repository does not explicitly state a license in the provided README.

Limitations & Caveats

The project is described as "AlphaGo Moment for Model Architecture Discovery" and the paper is "Coming Soon," suggesting it may be in an early or experimental stage. The core pipeline execution is demonstrated with a single evolution cycle, and the full scope of its autonomous capabilities requires further investigation.