This repository provides "Luotuo," a Chinese instruction-tuned LLaMA model, aimed at researchers and developers working with Chinese large language models. It offers a finetuned LLaMA model based on the Alpaca dataset, translated into Chinese, with the goal of improving Chinese language understanding and generation capabilities.

How It Works

Luotuo is built by applying LoRA (Low-Rank Adaptation) finetuning to the LLaMA base model. The core innovation lies in translating the Stanford Alpaca instruction dataset into Chinese, creating a Chinese-specific instruction-following dataset. This approach allows for efficient adaptation of LLaMA to Chinese, leveraging the original model's capabilities while specializing it for the target language.

Quick Start & Requirements

• Install/Run: The README mentions a Colab link for quick evaluation and a Gradio bot interface. Specific installation commands are not detailed, but it's based on HuggingFace's transformers library.
• Prerequisites: Requires LLaMA weights (not provided), Python, and standard ML libraries. GPU acceleration is essential for training and efficient inference.
• Resources: Training the 0.3 version took 7 hours and cost over $10 on unspecified hardware.

Highlighted Details

• Offers multiple model versions (0.1, 0.3) with varying training data and performance improvements.
• Training data includes translated Alpaca dataset and plans to incorporate Guanaco and other Chinese datasets.
• Provides evaluation code and an interactive chatbot interface.
• The project aims to become a central repository for various Chinese LLM projects, including CamelBell and Loulan.

Maintenance & Community

The project is actively developed by researchers from SenseTime and Huazhong Normal University. Community enthusiasm has led to expanded plans beyond the initial scope. Sponsorships are accepted to fund further development, data annotation, and computing power. A TODO list is maintained for future tasks.

Licensing & Compatibility

The repository itself appears to be under a permissive license, but it relies on LLaMA weights, which have their own usage restrictions. Compatibility with commercial or closed-source applications depends heavily on the LLaMA license terms.

Limitations & Caveats

The project is experimental, with ongoing development and planned improvements (e.g., tokenizer issues, larger datasets). The README notes that the training code is still being cleaned up. Performance claims are based on qualitative examples and specific evaluations, not comprehensive benchmarks.