This repository provides an open-source, commercially viable Chinese version of the Llama 2 7B model, along with a bilingual SFT dataset. It aims to offer a readily usable and adaptable Chinese language model for developers and researchers, enabling integration into various applications and further fine-tuning.

How It Works

The project fine-tunes the Llama 2 7B base model using a 10 million-entry bilingual SFT dataset. It adheres strictly to the Llama-2-chat input format, ensuring compatibility with existing optimizations and tools designed for the original Llama 2 chat model. This approach allows for seamless integration and leverages the established ecosystem around Llama 2.

Quick Start & Requirements

• Installation: Use transformers library for Python.
• Prerequisites: Python, PyTorch, CUDA (for GPU acceleration).
• Demo: Available via HuggingFace Spaces and Colab notebooks (FP16/INT4 require high RAM).
• Docker: A Dockerfile is provided for containerized deployment.
• GGML/CPU: Scripts and Docker images are available for converting and running models on CPU.
• API: A FastAPI-based API is included for deployment.
• Training: A detailed torchrun command is provided for fine-tuning.
• Resources: Links to HuggingFace, ModelScope, Baidu Netdisk, and GGML model downloads are provided.

Highlighted Details

• Fully open-source and commercially usable Chinese Llama 2 7B model.
• Includes a 10 million-entry bilingual SFT dataset.
• Supports 4-bit quantization (GGML) for CPU inference.
• Compatible with Llama-2-chat optimizations and ecosystem.
• Extensions include bilingual speech-to-text (LLaSM) and vision-to-text (LLaVA) models.

Maintenance & Community

• Active development with frequent updates (e.g., adding ModelScope links, GGML models, API deployment).
• WeChat group available for community exchange.

Licensing & Compatibility

• License: Apache-2.0.
• Compatibility: Permissive license allows for commercial use and integration into closed-source projects.

Limitations & Caveats

The Colab demos require high RAM and may not be usable on free tiers. The project is actively updated, implying potential for breaking changes in future releases.