X—LLM is a Python library designed to simplify and accelerate the fine-tuning of Large Language Models (LLMs) for engineers and researchers. It integrates cutting-edge optimization techniques like QLoRA, DeepSpeed, and Flash Attention 2, allowing users to focus on data and model improvements rather than complex implementation details.

How It Works

X—LLM employs a configuration-driven approach, abstracting complex training pipelines into a unified Config object. This allows users to easily toggle advanced features such as LoRA/QLoRA, 4-bit quantization, gradient checkpointing, and Flash Attention 2. The library handles the underlying integration with libraries like bitsandbytes, transformers, and deepspeed, streamlining the process of setting up efficient training runs.

Quick Start & Requirements

• Install: pip install xllm or pip install "xllm[train]" for training dependencies.
• Requirements: Python 3.8+, PyTorch 2.0.1+, CUDA 11.8+.
• Example: X—LLM Prototyping Notebook

Highlighted Details

• Supports numerous Hugging Face Transformers models (Yi-34B, Mistral, Llama 2, etc.).
• Integrates QLoRA, DeepSpeed, GPTQ, Flash Attention 2, FSDP, and gradient checkpointing.
• Seamlessly pushes checkpoints to HuggingFace Hub and reports to Weights & Biases.
• Offers built-in capabilities for fusing LoRA weights and GPTQ quantization.

Maintenance & Community

• Developed by Boris Zubarev.
• Community support via Discord: https://discord.gg/5znbxBgwZP

Licensing & Compatibility

• License: MIT.
• Compatible with commercial use and closed-source projects.

Limitations & Caveats

The library currently primarily supports the SODA dataset for demonstration, with plans to add more. Users are responsible for data preparation and handling. There's an open issue regarding DeepSpeed Stage 3 compatibility with bitsandbytes.