Yuan 2.0 is a family of open-source large language models developed by Inspur Information, offering models in 2B, 51B, and 102B parameter sizes. It aims to provide enhanced capabilities in semantics, mathematics, reasoning, and code generation, targeting researchers and developers for further development and application building.

How It Works

Yuan 2.0 builds upon its predecessor by utilizing a more diverse and high-quality dataset for pre-training and instruction fine-tuning. This approach aims to improve the model's understanding across various domains, including language, math, reasoning, and code. The project provides scripts for pre-training, fine-tuning, and inference services, enabling users to adapt and deploy the models for specific applications.

Quick Start & Requirements

• Installation: Docker is recommended.

  docker pull yuanmodel/yuan2.0:V1-base
  docker run --gpus all --privileged --ulimit stack=68719476736 --shm-size=1000G -itd -v /path/to/yuan_2.0:/workspace/yuan_2.0 -v /path/to/dataset:/workspace/dataset -v /path/to/checkpoints:/workspace/checkpoints --name your_name yuanmodel/yuan2.0:V1-base
  docker exec -it your_name bash
  

• Prerequisites: NVIDIA GPU with sufficient VRAM (2B: 8GB, 512B: 4x80GB, 102B: 8x80GB for inference). CUDA is implied for GPU usage.
• Resources: Significant GPU memory is required, especially for larger models.
• Documentation: Quick Start, Inference Guide

Highlighted Details

• Offers Hugging Face compatible versions of all models.
• Recent updates include improved coding and math capabilities for the 2B model (Yuan2-2B-Mars-hf) and GGUF support for CPU/iGPU inference.
• Supports deployment via vLLM and TensorRT-LLM & Triton Server.
• Integrates with FastChat for easier dialogue application building.
• Performance benchmarks show significant speedups (up to 4.37x) with TensorRT-LLM optimization for the 2B model.

Maintenance & Community

• Active development with frequent model updates (e.g., new 2B versions released in Jan, Feb, Apr 2024).
• "Yuan Large Model Co-training Plan" invites community input for model enhancement.
• Contact: air_service@ieisystem.com, WeChat group available via public account "源AI看世界".

Licensing & Compatibility

• Licensed under Apache 2.0.
• Permits commercial use without explicit authorization, subject to terms outlined in the "Yuan 2.0 Model License Agreement". Restrictions apply against use for harmful purposes or unapproved services.

Limitations & Caveats

• While efforts are made for data compliance, the project disclaims responsibility for output accuracy, potential misuse, or security/舆情 (public opinion) risks due to the model's scale and probabilistic nature. Users bear sole responsibility for model usage.