LLamaTuner is an open-source toolkit designed for efficient fine-tuning and deployment of Large Language Models (LLMs) and Vision-Language Models (VLMs). It supports a wide array of popular LLM architectures and offers various training methodologies, making it suitable for researchers and developers looking to customize models on diverse hardware, including single consumer GPUs.

How It Works

LLamaTuner leverages advanced techniques like QLoRA for memory-efficient fine-tuning, enabling training of large models on limited hardware. It integrates high-performance operators such as FlashAttention and Triton kernels to boost training throughput. The toolkit also supports DeepSpeed for distributed training and ZeRO optimizations, allowing for scalable fine-tuning of models exceeding 70B parameters across multiple nodes. Its flexible data pipeline accommodates various dataset formats for continuous pre-training, instruction fine-tuning, and agent fine-tuning.

Quick Start & Requirements

• Install: pip install -e . (after cloning the repository)
• Prerequisites: Python >= 3.10, PyTorch >= 2.2.0, Transformers >= 4.41.0, Datasets >= 2.19.1, Accelerate >= 0.30.1, PEFT >= 0.11.1, TRL >= 0.8.6. Optional: CUDA >= 12.2, DeepSpeed >= 0.14.0, bitsandbytes >= 0.43.1, vLLM >= 0.4.2, flash-attn >= 2.5.8.
• Hardware: QLoRA fine-tuning of a 7B model requires as little as 6GB VRAM (4-bit quantization). Full fine-tuning of a 70B model requires ~1200GB VRAM.
• Links: Getting Started, Supported Models, Data Preprocessing

Highlighted Details

• Supports fine-tuning over 30 LLM architectures including Llama 3, Mistral, Qwen, and Phi-3.
• Offers a comprehensive suite of training approaches: Pre-training, Supervised Fine-Tuning (SFT), Reward Modeling, PPO, DPO, KTO, and ORPO.
• Includes support for Vision-Language Models (VLMs) like LLaVA.
• Provides a model zoo with pre-trained adapters and inference capabilities.

Maintenance & Community

The project is actively maintained by jianzhnie and acknowledges contributions from the Hugging Face team and various open-source projects. Community engagement is encouraged via WeChat.

Licensing & Compatibility

LLamaTuner is released under the Apache 2.0 license, which permits commercial use and modification.

Limitations & Caveats

While the toolkit is highly efficient, achieving optimal performance may require specific hardware configurations and careful dependency management. Some datasets may require Hugging Face Hub login for access.