This repository provides code and methodologies for fine-tuning large language models (LLMs) using the Platypus approach, which leverages LoRA and PEFT for efficient refinement. It targets researchers and developers aiming to enhance LLM performance with custom datasets, offering a streamlined pipeline for data preparation, fine-tuning, and model merging.

How It Works

Platypus fine-tuning utilizes Low-Rank Adaptation (LoRA) and the Parameter-Efficient Fine-Tuning (PEFT) library to adapt base LLMs like LLaMA and LLaMa-2. The process involves a data pipeline that refines open-source datasets through keyword search and cosine similarity checks to remove redundancy and ensure data quality. Fine-tuning is orchestrated via finetune.py using torchrun for data parallelism or accelerate for broader compatibility, with hyperparameter configurations detailed for various model sizes. Merging LoRA weights back into the base model is handled by merge.sh.

Quick Start & Requirements

• Install dependencies: pip install -r requirements.txt (PyTorch 2.0+ recommended for torchrun).
• Fine-tuning requires PyTorch.
• For inference, clone the FastChat repository and install: pip install -e .
• Official project website: https://platypus-llm.github.io
• Fine-tuned models and datasets are available on HuggingFace: garage-bAInd

Highlighted Details

• Fine-tuning scripts support multi-GPU setups with data or model parallelism.
• Dataset refinement pipeline includes similarity checks to optimize training data.
• Benchmarking is supported via the LM Evaluation Harness, with example commands provided.
• Merging LoRA weights is facilitated by merge.sh using PEFT's linear merge.

Maintenance & Community

• The project is associated with Ariel N. Lee, Cole J. Hunter, and Nataniel Ruiz.
• Updates are provided periodically, with recent notes on LLaMa-2 7B fine-tuning and data pipeline improvements.

Licensing & Compatibility

• The repository itself does not explicitly state a license in the README.
• Models are available via HuggingFace, and their specific licenses should be consulted.

Limitations & Caveats

• Specific fine-tuning configurations (e.g., bf16=True, fp16=False) are noted as critical for LLaMa-2 7B.
• Users may need to manually adjust fine-tuning scripts if not using torchrun or PyTorch.
• Merging may require reinstalling bitsandbytes and peft from source.