This repository provides comprehensive materials for a full-day workshop on training and fine-tuning Large Language Models (LLMs), targeting data scientists and ML engineers. It offers hands-on notebooks and presentations covering essential LLM concepts, from basic embeddings and prompt engineering to advanced techniques like parameter-efficient fine-tuning (PEFT) and reinforcement learning from human feedback (RLHF).

How It Works

The workshop is structured into five modules, progressing from foundational knowledge to advanced alignment techniques. It utilizes Hugging Face's Transformers and PEFT libraries, demonstrating practical applications with models like Phi-3 Mini, Llama 3.1, and GPT-2. The approach emphasizes hands-on coding within Jupyter notebooks, complemented by conceptual explanations in presentation slides, enabling participants to build and adapt LLMs for various tasks.

Quick Start & Requirements

• Installation: Each module includes a *_Install_Requirements.ipynb notebook detailing necessary library installations.
• Environment: Recommended environment: PyTorch 2.4.0, Python 3.11, CUDA 12.4, Ubuntu 22.04.
• Hardware: A GPU with at least 48GB VRAM (e.g., NVIDIA A40) is recommended for fine-tuning tasks. A 30GB disk volume is needed for storing LLM weights.
• Resources: Links to module-specific setup notebooks are provided within the README.

Highlighted Details

• Covers prompt engineering with local LLMs (Phi-3 Mini, Llama 3.1).
• Demonstrates Parameter-Efficient Fine-Tuning (PEFT) techniques like QLoRA.
• Includes building custom Retrieval-Augmented Generation (RAG) pipelines.
• Explores alignment methods: RLHF, PPO, DPO, and ORPO.

Maintenance & Community

• The repository is maintained by Dipanjan (DJ) Sarkar.
• References Hugging Face documentation and various blogs for foundational concepts and implementation details.

Licensing & Compatibility

• The repository itself is hosted on GitHub, implying a standard open-source license, though not explicitly stated in the provided text.
• Content is intended for educational purposes within the workshop context.

Limitations & Caveats

• The workshop environment is optimized for specific hardware (48GB VRAM GPU) and software versions, which may require adjustments for different setups.
• Some exercises might require downloading large model weights (e.g., Llama 3), necessitating significant disk space.