Firefly is a comprehensive, one-stop training tool for large language models, designed for researchers and developers looking to pre-train, fine-tune, and apply Direct Preference Optimization (DPO) to a wide array of popular open-source LLMs. It offers efficient training methods like LoRA and QLoRA, significantly reducing resource requirements and accelerating the fine-tuning process, making advanced LLM customization accessible even with limited hardware.

How It Works

Firefly supports full parameter training, LoRA, and QLoRA, enabling users to choose the most suitable method based on their available resources. It integrates with Unsloth for accelerated training and reduced memory usage, boasting up to 47.32% speedup and 39.13% memory savings. The tool aligns fine-tuning templates with various official chat models, ensuring compatibility and consistent performance across a broad spectrum of LLMs.

Quick Start & Requirements

• Installation: pip install requirements.txt
• Unsloth Integration: pip install git+https://github.com/unslothai/unsloth.git and other specified dependencies.
• Prerequisites: PyTorch (version compatibility noted for specific models like Qwen1.5 and Gemma), Transformers, bitsandbytes, peft, xformers. CUDA is implicitly required for GPU acceleration.
• Resources: QLoRA training for Llama3-8B requires as little as 7.75GB VRAM.
• Documentation: Firefly GitHub Repository

Highlighted Details

• Supports pre-training, SFT, and DPO for numerous LLMs including Llama3, Qwen2, Yi-1.5, and Gemma.
• Achieved strong performance on the Open LLM Leaderboard with QLoRA-trained models, e.g., firefly-mixtral-8x7b scoring 70.16.
• Open-sourced several fine-tuned models and curated large-scale instruction datasets (e.g., firefly-train-1.1M).
• Offers LongQLoRA for efficient context length extension.

Maintenance & Community

• Active development with recent updates including Unsloth integration for Qwen2.
• Community support via WeChat groups and a Zhihu account.
• Huggingface Repo

Licensing & Compatibility

• The repository itself is likely under a permissive license (e.g., MIT, Apache 2.0, based on common practice, though not explicitly stated in the provided text).
• Users must adhere to the base model's commercial license.
• Open-sourced models may have limitations regarding harmful content generation.

Limitations & Caveats

Open-sourced models may produce factual inaccuracies, biased, or unethical content due to a lack of specific safety fine-tuning. Capabilities in coding and reasoning are noted as areas for improvement. Use is restricted to non-harmful purposes.