This repository provides code for fine-tuning Large Language Models (LLMs) using a curated quotes dataset. It targets users familiar with LLM fine-tuning who need a streamlined process, offering DeepSpeed, LoRA, and QLoRA methods.

How It Works

The project leverages Hugging Face's transformers library and integrates DeepSpeed for distributed training, enabling efficient fine-tuning of large models. LoRA and QLoRA are included for parameter-efficient fine-tuning, reducing memory requirements and training time. The code is adapted from an existing repository to support a wider range of models and methods.

Quick Start & Requirements

• Install/Run: Use the provided build_image.sh and run_image.sh scripts for Docker-based setup.
• Prerequisites:
  • NVIDIA GPU (>= 24GB VRAM, fp16 support recommended; A100 suggested).
  • Linux OS (Ubuntu recommended).
  • Modern Docker and NVIDIA Docker.
  • Latest NVIDIA drivers and CUDA drivers.
• Setup: Building the Docker image can take several minutes.
• Docs: Original Repo, Huggingface Model Walkthrough

Highlighted Details

• Supports DeepSpeed, LoRA, and QLoRA fine-tuning methods.
• Includes a formatted dataset (quotes_dataset) for fine-tuning.
• Dockerized workflow simplifies setup and dependency management.
• Original code adapted for broader model and method compatibility.

Maintenance & Community

• The repository mentions paid professional assistance is available via email.
• Links to older video walkthroughs are provided, but users are advised to use modern methods.

Licensing & Compatibility

• The repository does not explicitly state a license.

Limitations & Caveats

The project's Docker images may require updates for newer CUDA versions if build errors occur. Older video walkthroughs are not recommended for current usage.