self-adaptive-llms  by SakanaAI

Self-adaptation framework for real-time LLM adaptation

created 8 months ago
1,131 stars

Top 34.6% on sourcepulse

GitHubView on GitHub
1 Expert Loves This Project
Project Summary

This project introduces Transformer², a self-adaptation framework for Large Language Models (LLMs) designed to address the limitations of traditional, static fine-tuning. It enables LLMs to adapt to unseen tasks in real-time, offering a more dynamic and efficient approach for researchers and developers working with diverse NLP applications.

How It Works

Transformer² adapts LLMs by selectively adjusting singular components of their weight matrices, a novel approach that reduces computational overhead compared to full fine-tuning. During inference, a two-pass mechanism is employed: a dispatch system first identifies task properties, and then task-specific "expert" vectors, trained via reinforcement learning, are dynamically mixed to achieve targeted behavior for incoming prompts. This method allows for efficient, real-time adaptation without retraining the entire model.

Quick Start & Requirements

  • Installation: Clone the repository, create a conda environment (python=3.11), activate it, and install dependencies via pip install -r requirements.txt. The task evaluator requires an additional pip install -e . from within the evaluation/fishfarm directory.
  • Prerequisites: Python 3.11, Conda.
  • Usage: Example scripts for training (scripts/train_task_expert.sh) and evaluation (scripts/eval_prompt_based.sh, scripts/eval_few_shot.sh) are provided. Specific model and task configurations are set via script arguments.
  • Resources: No specific hardware requirements (e.g., GPU) are mentioned in the README, but LLM tasks typically benefit from GPU acceleration.

Highlighted Details

  • Introduces a novel "self-adaptation" mechanism for LLMs.
  • Employs a two-pass inference strategy with a dispatch system and dynamically mixed expert vectors.
  • Utilizes reinforcement learning for training task-specific expert vectors.
  • Offers scripts for both training and prompt-based/few-shot evaluation.

Maintenance & Community

The project is associated with SakanaAI. Further community engagement channels (e.g., Discord, Slack) or roadmap details are not explicitly provided in the README.

Licensing & Compatibility

The project's license is not specified in the README. Compatibility for commercial use or closed-source linking is therefore undetermined.

Limitations & Caveats

The README does not specify the license, which may impact commercial adoption. While the framework aims for real-time adaptation, the actual performance and resource requirements for different tasks are not detailed. The project appears to be research-oriented, and production-readiness is not explicitly stated.

Health Check
Last commit

6 months ago

Responsiveness

1 week

Pull Requests (30d)
0
Issues (30d)
1
Star History
91 stars in the last 90 days

Explore Similar Projects

Feedback? Help us improve.