LMFlow is an extensible toolkit for finetuning and inference of large foundation models, targeting researchers and developers aiming to customize and deploy LLMs efficiently. It provides a comprehensive suite of tools for model adaptation, optimization, and evaluation, simplifying the LLM lifecycle.

How It Works

LMFlow supports various finetuning methods including full finetuning, LoRA, and its novel memory-efficient LISA technique. It integrates advanced acceleration features like FlashAttention-2 and DeepSpeed Zero3 for optimized training. For inference, it supports streaming output, vLLM integration, and long-context capabilities via position interpolation.

Quick Start & Requirements

• Install via pip install lmflow-finetune or clone and install from source.
• Requires Python 3.9+ and mpi4py.
• Recommended for Linux or Google Colab.
• wandb is used for logging by default; can be disabled.
• Official documentation and examples are available.

Highlighted Details

• Supports a wide range of models including Llama, Qwen, Gemma, and more, with pre-configured conversation templates.
• Features LISA, a memory-efficient finetuning method outperforming LoRA in certain scenarios.
• Includes LMFlow Benchmark, an automatic evaluation framework for LLMs.
• Offers multimodal chatbot capabilities supporting image and text inputs.
• Supports custom optimizers for fine-tuning.

Maintenance & Community

The project has active development with frequent updates and contributions. Links to official documentation, model zoo, and a web service are provided.

Licensing & Compatibility

Licensed under Apache 2.0. Commercial use requires signing an authorization document.

Limitations & Caveats

Other OS platforms (macOS, Windows) are not fully tested and may encounter errors. The multimodal chatbot demo may experience queuing due to high demand.