OrionStar-Yi-34B-Chat is an open-source, bilingual (Chinese/English) conversational AI model fine-tuned from the Yi-34B base model by OrionStarAI. It aims to provide a high-quality, ChatGPT-like alternative for the open-source community, excelling in general domain benchmarks.

How It Works

This model is a fine-tuned version of the Yi-34B large language model, leveraging over 150,000 high-quality data samples. The fine-tuning process focuses on enhancing conversational capabilities and performance across various benchmarks, aiming for superior interaction experiences.

Quick Start & Requirements

• Install/Run: Use Hugging Face transformers library.

  from transformers import AutoModelForCausalLM, AutoTokenizer
  tokenizer = AutoTokenizer.from_pretrained("OrionStarAI/OrionStar-Yi-34B-Chat", trust_remote_code=True)
  model = AutoModelForCausalLM.from_pretrained("OrionStarAI/OrionStar-Yi-34B-Chat", device_map="auto", torch_dtype=torch.bfloat16, trust_remote_code=True)
  messages = [{"role": "user", "content": "你好! 你叫什么名字!"}]
  response = model.chat(tokenizer, messages, streaming=False)
  print(response)
  

• Prerequisites: Python, transformers, torch. device_map="auto" suggests utilizing available GPUs.
• Resources: Requires significant VRAM for the 34B parameter model.
• Links: Hugging Face, ModelScope, OpenXLab, Online Demo

Highlighted Details

• Achieves 77.71 on C-Eval, 78.32 on MMLU, and 73.52 on CMMLU (5-shot), outperforming many comparable models.
• Supports both Python code inference and a command-line interface (cli_demo.py).
• Demonstrates strong performance in conversational tasks, including complex reasoning like math word problems.
• Developed by OrionStarAI, a service robot solutions company with extensive AI experience.

Maintenance & Community

• Community: Discord
• Contact: ai@orionstar.com, 400-898-7779 for business inquiries.

Licensing & Compatibility

• License: Apache 2.0 and "Yi-34B Model Community License Agreement".
• Commercial Use: Requires adherence to both licenses; specific terms for commercial use should be reviewed.

Limitations & Caveats

The project statement includes a disclaimer of liability for any issues arising from the model's use, including data security, public opinion risks, or misuse, despite efforts to ensure data compliance. Users are warned against using the model for illegal activities or internet services without proper security review.