This repository provides a streamlined approach to fine-tuning large language models (LLMs) using the QLoRA technique. It targets AI researchers and developers looking to efficiently adapt pre-trained LLMs for specific tasks or datasets, enabling custom model creation with reduced computational resources.

How It Works

The project leverages QLoRA, a parameter-efficient fine-tuning method that quantizes pre-trained LLMs to 4-bit precision and uses Low-Rank Adaptation (LoRA) to fine-tune only a small subset of adapter weights. This significantly reduces memory requirements and computational cost during training while maintaining performance comparable to full fine-tuning.

Quick Start & Requirements

• Install dependencies: pip install -r requirements.txt
• Run training: python train.py <config_file> (e.g., python train.py configs/llama3_8b_chat_uncensored.yaml)
• Requires Python 3.8+.
• Official examples and Colab notebook available for inference.

Highlighted Details

• Supports fine-tuning of models like Llama3-8B and Llama2-7B.
• Includes pre-configured YAML files for various datasets and models.
• Provides instructions for pushing trained models to HuggingFace Hub.
• Offers guidance on converting models to GGUF format for use with llama.cpp.

Maintenance & Community

• The repository is maintained by georgesung.
• Links to HuggingFace models are provided.

Licensing & Compatibility

• The repository's license is not explicitly stated in the README.
• Compatibility for commercial use or closed-source linking is not specified.

Limitations & Caveats

Users on Python 3.7 may encounter compatibility issues with newer transformers and peft versions, requiring specific older package versions for stable operation.