This repository provides a collection of notebooks and guides for efficiently fine-tuning large language models (LLMs) using Parameter-Efficient Fine-Tuning (PEFT) techniques, primarily LoRA and QLoRA, with Hugging Face's transformers library. It targets researchers and developers looking to adapt LLMs for specific tasks or datasets without the prohibitive cost of full fine-tuning.

How It Works

The project leverages the PEFT library from Hugging Face, which enables efficient adaptation of LLMs by training only a small number of parameters. Techniques like LoRA (Low-Rank Adaptation) inject trainable low-rank matrices into the model's layers, significantly reducing computational and memory requirements. The notebooks demonstrate this approach across various popular LLMs, including Llama, Falcon, OPT, and Mistral, often incorporating quantization methods like BNB and QLoRA for further memory savings.

Quick Start & Requirements

• Install: Primarily uses Hugging Face libraries (transformers, peft, datasets, accelerate, bitsandbytes). Installation via pip is standard.
• Prerequisites: Python 3.8+, PyTorch, and often a GPU with sufficient VRAM (e.g., 16GB+ for 7B models with QLoRA). CUDA is recommended for GPU acceleration.
• Resources: Notebooks are provided with Colab badges, indicating they can be run in Google Colab, often with free GPU access.
• Docs: The README itself serves as a table of contents for the provided notebooks.

Highlighted Details

• Comprehensive coverage of LoRA and QLoRA fine-tuning for numerous LLMs.
• Includes notebooks for RLHF, RAG pipelines, and knowledge graph generation.
• Demonstrates fine-tuning with quantization (8-bit, 4-bit, 2-bit) for memory efficiency.
• Features examples for models like Llama 2, Falcon, Mistral, Phi-1.5, and GPT-NeoX.

Maintenance & Community

• The project is maintained by Ashish Patel.
• Contributions are welcomed via issues and pull requests.

Licensing & Compatibility

• Licensed under the MIT License.
• Permissive license allows for commercial use and integration into closed-source projects.

Limitations & Caveats

The project consists of individual notebooks, which may vary in their level of abstraction and error handling. Users should verify the compatibility of specific notebook code with their environment and chosen LLM versions. Some advanced techniques like RLHF or complex RAG pipelines may require significant computational resources and deeper understanding.