LitGPT provides over 20 high-performance Large Language Models (LLMs) with comprehensive recipes for pretraining, finetuning, and deployment. It targets developers and researchers seeking efficient, scalable, and customizable LLM solutions, offering a no-abstraction, beginner-friendly approach for enterprise-grade applications.

How It Works

LitGPT implements LLMs from scratch, prioritizing performance and minimal abstractions. It leverages PyTorch Lightning Fabric for distributed training across GPUs and TPUs, supporting advanced techniques like Flash Attention v2, Fully Sharded Data Parallelism (FSDP), and parameter-efficient finetuning methods (LoRA, QLoRA, Adapters). This design enables reduced memory usage through quantization (4-bit, 8-bit) and mixed-precision training (FP16, BF16), facilitating efficient operation on lower-memory GPUs and at scale.

Quick Start & Requirements

• Install: pip install 'litgpt[all]'
• Prerequisites: Python 3.8+, PyTorch. GPU recommended for training/inference.
• Usage: Load and generate text with from litgpt import LLM; llm = LLM.load("microsoft/phi-2"); llm.generate(...)
• More Info: Quick start, Models, Finetune, Deploy

Highlighted Details

• Supports 20+ LLMs including Llama, Code Llama, Gemma, Phi, Qwen, Mistral, and Falcon.
• Offers optimized workflows for pretraining, finetuning (LoRA, QLoRA), evaluation, and deployment.
• Features quantization (4-bit, 8-bit) and mixed-precision training for reduced memory footprint.
• Includes validated YAML configuration files for training recipes and CLI overrides.
• Powers projects like SAMBA, the NeurIPS 2023 LLM Efficiency Challenge, and TinyLlama.

Maintenance & Community

• Actively maintained by Lightning AI.
• Community support via Discord: https://discord.gg/VptPCZkGNa
• Open to contributions and feature requests.

Licensing & Compatibility

• Released under the Apache 2.0 license, permitting unlimited enterprise use.

Limitations & Caveats

Some model downloads may require an additional access token, as detailed in the documentation. The project is built upon Lightning Fabric, extending nanoGPT and Lit-LLaMA.