BlueLM is an open-source suite of large language models developed by vivo AI Lab, offering 7B parameter models with both base and chat variants. It addresses the need for high-performance, multilingual LLMs with extended context capabilities, targeting researchers and developers. The models provide competitive performance on various benchmarks and support commercial applications under a permissive license.

How It Works

BlueLM models are trained on a 2.6 trillion token multilingual corpus, including Chinese, English, and some Japanese/Korean data. The 7B models are available in standard 2K context length and extended 32K context length versions. Quantized versions (4-bit AWQ/GPTQ) are also provided for reduced VRAM requirements. The project emphasizes strong performance on benchmarks like C-Eval and CMMLU, aiming for competitive results against other open-source models.

Quick Start & Requirements

• Install dependencies via pip install -r requirements.txt.
• For 32K context models, flash_attn (version 2.3.3 or pre-compiled) is recommended.
• Inference can be performed using Hugging Face transformers library, with provided CLI, web (Streamlit), and OpenAI-compatible API server demos.
• Quantized model inference requires quant_cuda installation.
• vLLM inference requires NVIDIA driver 525.125.06 and CUDA 12.1.

Highlighted Details

• Offers 7B base and chat models, with 32K context length variants.
• Achieves strong results on C-Eval (72.7) and CMMLU (74.2) for the chat model.
• Supports function calling and provides an OpenAI-compatible API server.
• Includes 4-bit quantized models requiring ~5GB VRAM.
• Fine-tuning scripts for full fine-tuning and LoRA are available.

Maintenance & Community

• The project was updated on March 25, 2024, to include 32K context and function calling capabilities.
• The license was updated to the OpenAtom Foundation Model License on December 25, 2024.
• Contact is available via email (developers-ai@vivo.com) and a WeChat group.

Licensing & Compatibility

• Licensed under the OpenAtom Foundation Model License (effective Dec 25, 2024).
• The license permits commercial use, modification, and distribution with fewer restrictions than the previous vivo_BlueLM model license.

Limitations & Caveats

The project statement strongly advises against using the models for activities that harm national social security or violate laws, and recommends thorough risk assessment for product applications. The developers disclaim responsibility for any issues arising from the use of the open-source models.