OpenFedLLM provides a research-oriented codebase for training Large Language Models (LLMs) using federated learning (FL) on decentralized private data. It targets researchers and practitioners interested in privacy-preserving LLM development, offering a framework to implement and evaluate various FL algorithms alongside LLM training techniques like instruction tuning and direct preference optimization.

How It Works

The framework integrates 7 distinct federated learning algorithms (e.g., FedAvg, FedProx, SCAFFOLD) with 2 LLM training methods: Supervised Fine-Tuning (SFT) and Direct Preference Optimization (DPO). It supports extensive evaluation across 30+ metrics, covering diverse capabilities like medical QA, code generation, and mathematical reasoning. The modular design allows for easy integration of new FL algorithms and datasets, facilitating empirical studies on privacy-preserving LLM training.

Quick Start & Requirements

• Install: Clone the repository with submodules (git clone --recursive --shallow-submodules), create a conda environment (conda create -n fedllm python=3.10), activate it (conda activate fedllm), and install requirements (pip install -r requirements.txt).
• Prerequisites: Python 3.10, CUDA (implied for LLM training), and potentially large datasets for meaningful FL.
• Setup: Run source setup.sh.
• Resources: Training LLMs, even with PEFT, requires significant GPU resources. Specific hardware requirements are not detailed but are substantial.
• Docs: Paper accepted by KDD 2024, FedLLM-Bench accepted by NeurIPS 2024 Datasets and Benchmarks Track.

Highlighted Details

• Implements 7 federated learning algorithms and 2 LLM training methods (SFT, DPO).
• Supports 30+ evaluation metrics across various domains.
• Includes FedLLM-Bench, a realistic benchmark for federated LLM research.
• Leverages PEFT (LoRA) for efficient LLM fine-tuning.

Maintenance & Community

The project is associated with authors from institutions like Peking University and is accepted at KDD 2024 and NeurIPS 2024 Datasets and Benchmarks Track, indicating academic backing. Community channels are not explicitly mentioned in the README.

Licensing & Compatibility

The repository is released under an unspecified license. Compatibility for commercial use or closed-source linking is not detailed.

Limitations & Caveats

The README indicates that templates for DPO training are defined separately from SFT, suggesting potential inconsistencies. Support for custom datasets requires modification of utils/process_dataset.py. The project is research-focused, and production-readiness is not guaranteed.