ExpertLLaMA is an open-source chatbot designed to produce high-quality, expert-like responses by augmenting instructions with specialized "Expert Identity" descriptions. It targets researchers and developers seeking to enhance instruction-following capabilities in LLMs, offering a cost-effective alternative to models trained on more expensive data.

How It Works

The core innovation is "ExpertPrompting," which involves generating customized expert identity descriptions for specific instructions. These descriptions are then prepended to the original instructions, creating augmented training data. This approach aims to elicit better performance from LLMs by guiding them to adopt specific expert personas, leading to more elaborate and accurate responses compared to standard instruction tuning.

Quick Start & Requirements

• Installation: Apply delta weights to official LLaMA weights using python3 apply_delta.py.
• Prerequisites: Official LLaMA model weights (7B), Python 3.x.
• Inference: Requires approximately 15GB memory (fp16).
• Demo: Live demo available at Hugging Face Space.

Highlighted Details

• Achieves 96% of ChatGPT's capability based on GPT-4 evaluation.
• Outperforms models like Vicuna and LLaMA-GPT4.
• Trained on 52k instruction-following expert data generated by GPT-3.5-turbo.
• Delta weights are released to comply with LLaMA license.

Maintenance & Community

• Initial release in May 2023.
• Paper available: "ExpertPrompting: Instructing Large Language Models to be Distinguished Experts."

Licensing & Compatibility

• Data is licensed under CC BY-NC 4.0 (non-commercial use only).
• Models trained using the dataset are restricted to research purposes.

Limitations & Caveats

The dataset and models are strictly for research use and cannot be employed for commercial applications due to the CC BY-NC 4.0 license.