XTuner is a comprehensive toolkit for fine-tuning large language models (LLMs) and visual-language models (VLMs), designed for efficiency and flexibility. It supports a wide array of models including InternLM, Llama, Mistral, Qwen, and Phi, catering to researchers and developers needing to adapt these models for specific tasks. XTuner enables efficient fine-tuning techniques like QLoRA and full-parameter tuning, even on limited hardware, and integrates with popular distributed training frameworks like DeepSpeed.

How It Works

XTuner leverages optimized kernels (FlashAttention, Triton) and DeepSpeed integration for high-throughput training. Its architecture supports various fine-tuning methods (QLoRA, LoRA, full-parameter) and data processing pipelines, allowing users to customize training from continuous pre-training to instruction and agent fine-tuning. It also facilitates multi-modal VLM pre-training and fine-tuning using architectures like LLaVA.

Quick Start & Requirements

• Installation: pip install -U xtuner or pip install -U 'xtuner[deepspeed]'. Source install: git clone https://github.com/InternLM/xtuner.git && cd xtuner && pip install -e '.[all]'.
• Prerequisites: Python 3.10+. Supports fine-tuning 7B LLMs on 8GB GPUs, with multi-node support for larger models.
• Resources: Fine-tuning examples include QLoRA for InternLM2.5-Chat-7B on a single GPU or multi-GPU setups.
• Docs: Usage, Speed Benchmark, Chat, Deployment.

Highlighted Details

• Supports a broad range of LLMs (InternLM, Llama 2/3, Mistral, Qwen, Mixtral, DeepSeek V2, Gemma, Phi-3) and VLMs (LLaVA).
• Offers various fine-tuning algorithms: QLoRA, LoRA, Full parameter, DPO, ORPO, Reward Model training.
• Includes features for continuous pre-training, instruction fine-tuning, and agent fine-tuning.
• Seamless integration with deployment (LMDeploy) and evaluation (OpenCompass, VLMEvalKit) toolkits.

Maintenance & Community

• Active development with frequent updates supporting new models and techniques.
• Community channels: WeChat, Twitter, Discord.
• Models available on Hugging Face, ModelScope, OpenXLab, and WiseModel.

Licensing & Compatibility

• Released under Apache License 2.0. Users must also adhere to the licenses of the models and datasets used.

Limitations & Caveats

The project's rapid development pace means new models and features are frequently added, potentially leading to breaking changes or requiring frequent updates to dependencies. Specific hardware requirements may vary significantly based on the model size and fine-tuning method employed.