DevOps-Models is a series of open-source LLMs tailored for the DevOps domain, designed to assist engineers with questions across the entire DevOps lifecycle. Built upon the Qwen series, these models are fine-tuned on high-quality Chinese DevOps corpora and QA datasets, aiming to achieve state-of-the-art performance among models of similar scale.

How It Works

The models leverage the Qwen architecture, enhanced through a two-stage fine-tuning process. First, a Base model is trained on a curated dataset of Chinese DevOps-related text. Subsequently, a Chat model is developed by aligning the Base model with DevOps-specific question-answering data. This approach aims to provide accurate and contextually relevant responses for DevOps-related queries.

Quick Start & Requirements

• Install dependencies using pip install -r requirements.txt.
• Utilize Hugging Face Transformers for inference.
• Example inference code and model download links for 7B and 14B parameter versions (Base and Chat) are provided.
• Online demo available at: https://modelscope.cn/studios/codefuse-ai/DevOps-Model-Demo/summary

Highlighted Details

• Achieves top scores on selected DevOps-related benchmarks from CMMLU and CEval datasets in both zero-shot and five-shot settings.
• Offers both Base and Chat model variants for different use cases.
• Supports custom model training and quantization (GPTQ).
• Active development with new model releases and an ongoing effort to build a dedicated DevOps evaluation benchmark (DevOpsEval).

Maintenance & Community

• Developed by Ant Group's Platform Technology Business Group Risk Intelligence team and Peking University.
• Model updates and releases are announced on Hugging Face and ModelScope.
• Links to Hugging Face and ModelScope model repositories are provided.

Licensing & Compatibility

• The README does not explicitly state a license.
• The disclaimer mentions that service providers using the model commercially are responsible for any adverse effects or harmful speech generated.

Limitations & Caveats

• The models may generate hallucinations or discriminatory remarks, requiring cautious use.
• Commercial use requires the service provider to assume responsibility for any negative outcomes.
• Int4 quantized versions are listed as "Coming Soon."