This repository provides a comprehensive tutorial for users to master the end-to-end workflow of Llama 3, covering fine-tuning, quantization, deployment, and evaluation. It is designed for developers and researchers looking to leverage Llama 3's capabilities through the Shanghai AI Laboratory's Pudur (浦语) large model toolchain.

How It Works

The tutorial guides users through practical applications of Llama 3 using key components of the Pudur toolchain: XTuner for fine-tuning, LMDeploy for efficient deployment, and OpenCompass for model evaluation. This integrated approach simplifies complex LLM operations, offering a structured learning path from basic deployment to advanced agent capabilities and performance benchmarking.

Quick Start & Requirements

• Primary install/run: Follow module-specific instructions within the docs/ or docs_autodl/ directories.
• Prerequisites: VScode remote connection to development machines is recommended. Specific hardware requirements (e.g., GPU, VRAM) will vary by module and are detailed in the documentation.
• Resources: Links to detailed documentation and video tutorials are provided for each section.

Highlighted Details

• Covers local web demo deployment of Llama 3.
• Includes fine-tuning for personal assistants and image understanding (LLaVA).
• Demonstrates efficient deployment with LMDeploy and agent capabilities.
• Features model evaluation using OpenCompass.

Maintenance & Community

This project is associated with the SmartFlowAI and Pudur (浦语) communities. Users are encouraged to join discussion groups for Llama 3. Computing resources were supported by Pudur community A100 instances.

Licensing & Compatibility

The repository itself does not explicitly state a license. However, it heavily relies on and links to other projects (XTuner, LMDeploy, OpenCompass), which have their own licenses. Users should verify the licensing terms of these underlying components for compatibility, especially for commercial use.

Limitations & Caveats

The tutorial assumes familiarity with remote development environments like VScode. Specific hardware requirements for each module are not consolidated in the main README and must be checked in individual section documentation.