Summary Shepherd addresses the cost and privacy challenges of collecting instruction data for LLM fine-tuning. It provides a foundational Federated Learning (FL) framework for instruction tuning, enabling researchers and practitioners to create personalized LLMs using diverse, user-specific datasets without compromising privacy. The framework prioritizes ease of use, adaptability, and scalability for exploring LLM personalization.

How It Works The approach uses Federated Learning to train LLMs across distributed clients with heterogeneous instruction datasets. Parameter-efficient fine-tuning (PEFT) via LoRA is employed for efficient local training, built upon Hugging Face's PEFT and bitsandbytes libraries. This methodology facilitates the integration of novel algorithms and configurations for advanced research.

Quick Start & Requirements Requires Python 3.8. Installation: pip install -r requirements.txt. Note potential bitsandbytes installation issues, with specific instructions for Windows. Data preparation involves python client_data_allocation.py <num_client> <diff_quantity>, defaulting to a version of databricks-dolly-15k. Federated fine-tuning is initiated via python main.py, and inference via GlobalModel_generate.py. Links to the associated paper and GitHub repo are provided.

Highlighted Details

• Local training with LoRA completes within hours on a single NVIDIA TITAN RTX.
• Supports popular LLMs like LLaMA, Alpaca, Vicuna, and Baize.
• Designed for adaptability, scalability, and seamless integration of new algorithms.
• Enables training on diverse, heterogeneous instruction datasets across clients.

Maintenance & Community The project is actively enhanced with bug fixes and new features. Developers welcome pull requests for research goals, algorithmic improvements, and hardware simulation. No specific community channels are listed.

Licensing & Compatibility The data, code, and checkpoints are licensed strictly for "research use only." This license prohibits commercial applications or integration into closed-source projects without explicit permissions.

Limitations & Caveats The primary limitation is the "research use only" license, restricting adoption to non-commercial research. Potential installation complexities with bitsandbytes and the use of an older databricks-dolly-15k dataset may require additional setup effort.