Axolotl is a comprehensive toolkit for streamlining the post-training of large language models, targeting AI researchers and engineers. It simplifies complex fine-tuning tasks like LoRA, QLoRA, and full fine-tuning, enabling efficient customization of pre-trained models.

How It Works

Axolotl leverages a YAML configuration system to manage the entire model training pipeline, from data preprocessing to inference and evaluation. This approach offers a unified and reproducible workflow. It integrates with performance-enhancing libraries such as xformers, Flash Attention, and various multi-GPU strategies (FSDP, DeepSpeed), aiming to maximize training speed and efficiency.

Quick Start & Requirements

• Installation: pip install --no-build-isolation axolotl[flash-attn,deepspeed]
• Prerequisites: NVIDIA GPU (Ampere+ recommended for bf16/Flash Attention), Python 3.11, PyTorch ≥2.4.1.
• Examples: Fetch example configurations via axolotl fetch examples.
• First Fine-tune: Run axolotl train examples/llama-3/lora-1b.yml.
• Documentation: Getting Started Guide

Highlighted Details

• Supports a wide range of Hugging Face models including LLaMA, Mistral, Mixtral, Falcon, and Pythia.
• Offers multiple training methods: full fine-tuning, LoRA, QLoRA, ReLoRA, and GPTQ.
• Features performance optimizations like Flash Attention, xformers, and multi-packing.
• Integrates with logging platforms like Weights & Biases, MLflow, and Comet.

Maintenance & Community

The project is sponsored by Modal. Community support is available via Discord.

Licensing & Compatibility

Licensed under the Apache 2.0 License, permitting commercial use and integration with closed-source projects.

Limitations & Caveats

Support for certain models (e.g., Mixtral-MoE, Falcon, Pythia) with specific optimizations like GPTQ or Flash Attention is marked as untested or not supported in the provided table.