Cabrita is a Portuguese-language instruction-tuned LLaMA model designed for research purposes. It addresses the need for Portuguese NLP capabilities by finetuning the LLaMA-7B model using a translated version of the Stanford Alpaca dataset. The project provides resources and steps for replicating the finetuning process, enabling researchers to explore Portuguese language model development.

How It Works

The project leverages the Alpaca Lora codebase for finetuning LLaMA. The Stanford Alpaca dataset was translated to Portuguese using ChatGPT, incurring minimal cost. This approach allows for efficient finetuning, with the authors reporting impressive results after just one hour of training on a single A100 GPU.

Quick Start & Requirements

• Finetuning can be performed using the provided Colab notebook.
• Requires LLaMA-7B model weights.
• A single A100 GPU is recommended for finetuning.
• The evaluation notebook is available for testing.

Highlighted Details

• Finetuned LLaMA-7B model with Portuguese instruction data.
• Dataset translation cost approximately US$8.00.
• Training completed in 1 hour on an A100 GPU.
• Provides example outputs comparing Cabrita to Stanford Alpaca.

Maintenance & Community

• Authors: piEsposito, pedrogengo.
• No community links (Discord/Slack) or roadmap are provided in the README.

Licensing & Compatibility

• The model is intended for research use only and cannot be used for commercial purposes.
• Specific license details beyond this restriction are not explicitly stated.

Limitations & Caveats

The dataset translation, while cost-effective, is noted as not being of the highest quality. The model is strictly for research use and prohibits commercial applications.