mLoRA is an open-source framework for efficiently fine-tuning multiple Large Language Models (LLMs) using LoRA and its variants. It targets researchers and practitioners needing to train numerous adapters concurrently, offering significant computational and memory savings by sharing a base model and employing pipeline parallelism.

How It Works

mLoRA utilizes a pipeline parallelism algorithm to distribute the fine-tuning process across multiple GPUs. This approach allows for concurrent training of multiple LoRA adapters, each sharing a common base LLM. The framework supports various LoRA variants (QLoRA, LoRA+, VeRA, DoRA) and preference alignment algorithms (DPO, CPO, CIT), enabling flexible and efficient multi-adapter fine-tuning.

Quick Start & Requirements

• Install: pip install . (Python >= 3.12) or use the official Docker image (yezhengmaolove/mlora:latest).
• Prerequisites: NVIDIA GPU, CUDA.
• Usage: python mlora_train.py --base_model <model_name> --config <config_path>
• Docs: [TODO] (Mentioned but not linked)
• Demo: Configuration files in demo/ folder.

Highlighted Details

• Achieves higher throughput than PEFT-LoRA with FSDP/TP on benchmarks (e.g., 2364 tokens/s vs. 1750 tokens/s for llama-2-7b).
• Supports multiple LoRA variants including QLoRA, LoRA+, VeRA, and DoRA.
• Integrates with preference alignment algorithms like DPO, CPO, and CIT.
• Offers a Dockerized deployment for service-based fine-tuning with a CLI client.

Maintenance & Community

• Accepted by VLDB'25.
• Contribution guidelines provided; encourages pull requests and issues.
• Pre-commit hooks available for code checking.

Licensing & Compatibility

• Licensed under the Apache 2.0 License.
• Permissive license suitable for commercial use and integration with closed-source projects.

Limitations & Caveats

The project mentions "Document Help Document[TODO]" and "How to develop a new adapter", indicating that some documentation may be incomplete or under development.