Fengshenbang-LM is an open-source ecosystem of large models developed by IDEA Institute, aiming to serve as foundational infrastructure for Chinese AI-generated content (AIGC) and cognitive intelligence. It offers a comprehensive suite of pre-trained models, fine-tuned applications, benchmarks, and datasets, catering to researchers and developers focused on Chinese NLP tasks.

How It Works

The project provides a diverse range of models categorized by task type: general (NLU, NLG, NLT), multimodal, and domain-specific. It leverages a foundational model approach, enabling adaptation to various downstream tasks with potentially reduced computational resources. The ecosystem emphasizes continuous upgrades, integrating the latest data and training algorithms to build a standardized, user-centric infrastructure for Chinese NLP.

Quick Start & Requirements

• Installation: pip install --editable . (after cloning the repository and initializing submodules). Docker is also provided.
• Prerequisites: PyTorch, CUDA (for GPU acceleration), and potentially Hugging Face libraries. Specific models may have varying hardware requirements.
• Resources: Training large models requires significant GPU resources. Model inference can be performed on consumer-grade hardware depending on model size.
• Links: Fengshenbang-LM GitHub, Huggingface Community

Highlighted Details

• Offers models like "Jiang Ziya" (general large models), "Taiyi" (multimodal, including Chinese Stable Diffusion), and "Erlangshen" (NLU, largest Chinese BERT model at release).
• Includes the "FengShen" framework for distributed training and fine-tuning, inspired by HuggingFace and Megatron-LM.
• Provides command-line pipelines for easy prediction and fine-tuning of various NLP tasks.
• Achieved state-of-the-art results on Chinese NLP benchmarks like FewCLUE and ZeroCLUE.

Maintenance & Community

The project is actively maintained by the IDEA Institute's CCNL team. Community engagement is encouraged through WeChat groups and ongoing recruitment.

Licensing & Compatibility

• License: Apache License 2.0.
• Compatibility: Permissive license suitable for commercial use and integration into closed-source projects.

Limitations & Caveats

While comprehensive, the project's focus is primarily on Chinese language tasks. Some models and documentation might be more mature for Chinese than English. The rapid evolution of large models means specific model versions may become outdated.