pandallm  by dandelionsllm

Open-source LLM project for Chinese language exploration

created 2 years ago
1,038 stars

Top 36.8% on sourcepulse

GitHubView on GitHub
1 Expert Loves This Project
Project Summary

PandaLLM is an open-source project focused on developing and deploying Chinese large language models. It offers a suite of tools and models for training, inference, and community collaboration, aiming to advance Chinese Natural Language Processing (NLP) research and application. The project targets developers and researchers interested in the full LLM technology stack.

How It Works

PandaLLM comprises three main components: PandaLLM (models), PandaLLMOps (tools), and PandaCommunity (community). PandaLLM models are primarily based on LLaMA and LLaMA2, with continuous pre-training on Chinese datasets. PandaLLMOps provides a unified framework for LLM lifecycle management, supporting pre-training, fine-tuning (full parameter, LoRA, QLoRA), and efficient deployment using engines like vLLM and LightLLM. The approach emphasizes using native PyTorch, DeepSpeed, and Huggingface Transformers for flexibility and ease of modification.

Quick Start & Requirements

  • PandaLLM Models: Available on Hugging Face. Specific model weights are provided as deltas for LLaMA1-based models, requiring a conversion script. LLaMA2-based models are directly available.
  • PandaLLMOps: Requires Python, DeepSpeed, and Huggingface Transformers. Training examples utilize Hydra for configuration.
  • Panda-Index: Can be deployed using Huggingface transformers.
  • Hardware: Training examples suggest significant GPU resources (e.g., 8x A100 80G). Inference can be optimized for lower-resource environments.
  • Documentation: Tutorials are available at https://panda-tutorial.readthedocs.io/en/latest/index.html.

Highlighted Details

  • Offers both LLaMA1 and LLaMA2-based Chinese LLMs, including specialized models like Code-Panda-13B-Python.
  • PandaLLMOps supports advanced training techniques like Pipeline Parallelism and LoRA/QLoRA, with detailed configuration examples.
  • Panda-Index models achieve top-3 performance on MTEB retrieval benchmarks for text embedding.
  • Provides tools for efficient inference, integrating vLLM and LightLLM.

Maintenance & Community

The project is actively developed with regular updates and releases. A strong emphasis is placed on community building through PandaCommunity, offering tutorials, technical discussions, and collaborative projects. Community engagement channels include WeChat groups and GitHub Issues.

Licensing & Compatibility

Models based on LLaMA2 and OpenLLaMA are noted as commercially usable. LLaMA1-based models have restrictions due to the original LLaMA license, with weights provided as deltas. The project's disclaimer requests users not to use the code, data, or models for commercial purposes (except for explicitly stated commercially usable versions) or harmful activities.

Limitations & Caveats

  • LLaMA1-based model weights require a conversion script due to licensing.
  • Training with bfloat16 on non-Ampere GPUs may require careful handling during fine-tuning.
  • The project notes potential training speed issues with OpenLLaMA models, possibly due to model precision.
  • ZeRO-3 is not recommended for training due to significant communication overhead and extended training times.
Health Check
Last commit

1 year ago

Responsiveness

1 day

Pull Requests (30d)
0
Issues (30d)
0
Star History
3 stars in the last 90 days

Explore Similar Projects

Starred by Chip Huyen Chip Huyen(Author of AI Engineering, Designing Machine Learning Systems), Jeff Hammerbacher Jeff Hammerbacher(Cofounder of Cloudera), and
3 more.

LLaMA-Adapter by OpenGVLab

0.0%
6k
Efficient fine-tuning for instruction-following LLaMA models
created 2 years ago
updated 1 year ago
Starred by Chip Huyen Chip Huyen(Author of AI Engineering, Designing Machine Learning Systems), Ying Sheng Ying Sheng(Author of SGLang), and
9 more.

alpaca-lora by tloen

0.0%
19k
LoRA fine-tuning for LLaMA
created 2 years ago
updated 1 year ago
Feedback? Help us improve.