OpenBuddy is an open-source, multilingual chatbot designed for global users, offering conversational AI capabilities across English, Chinese, Japanese, Korean, French, and German. It aims to provide a free, offline-capable AI model that can run on user devices, empowering individuals worldwide with accessible AI technology.

How It Works

OpenBuddy is fine-tuned from foundational models like Tii's Falcon and Facebook's LLaMA. It incorporates an extended vocabulary, additional common characters (especially CJK), and enhanced token embeddings. This approach, combined with multi-turn dialogue datasets, improves its performance in question answering and translation tasks across multiple languages.

Quick Start & Requirements

• Ollama (Recommended): ollama run openbuddy/openbuddy-llama3-8b-v21.1-8k
• vLLM: Requires Linux + CUDA GPU for high-concurrency inference.
• Dependencies: Python, CUDA (for vLLM), Ollama.
• Resources: Quantized versions (3/4/5-bit) are available for CPU deployment via llama.cpp.
• Demo: Available on their Discord server.
• Documentation: https://openbuddy.ai, https://github.com/OpenBuddy/OpenBuddy

Highlighted Details

• Supports various model sizes (3B to 70B+).
• Fine-tuned with multi-turn dialogue datasets.
• Offers 3/4/5-bit quantization for CPU deployment.
• vLLM integration provides OpenAI-compatible API service.

Maintenance & Community

• Active development plans include enhancing multilingual performance, optimizing quantized models, and exploring RLHF and multimodal capabilities.
• Community engagement via Discord and ModelScope.
• https://modelscope.cn/organization/OpenBuddy

Licensing & Compatibility

• OpenBuddy-LLaMA models are subject to Meta's LLaMA license agreement, requiring prior approval from Meta.
• OpenBuddy-Falcon, Mistral, Zephyr, and OpenLLaMA series models are released under Apache 2.0.
• Source code is under Apache 2.0. Commercial use is permitted for Apache 2.0 licensed components, but LLaMA-based models require Meta's approval.

Limitations & Caveats

Models are provided "as-is" and may produce erroneous or undesirable outputs. Users should avoid using them in critical or high-stakes situations. The LLaMA-based models have licensing restrictions tied to Meta's approval process.