This repository provides scripts to fine-tune LLaMa3 and other foundation models for non-English languages. It addresses the limitation of base models performing poorly in languages other than English by translating datasets and fine-tuning using QLoRA and PEFT. The target audience includes researchers and developers aiming to create multilingual LLMs.

How It Works

The process involves translating a base dataset (like OASST1) to a target language using various translation models (OPUS M2M, MADLAD, NLLB, etc.). These translated conversations are then structured into prompt threads. Finally, QLoRA and PEFT are used to fine-tune a foundation model on this translated data, with optional Direct Preference Optimization (DPO) or Ordinal Preference Optimization (ORPO) for further refinement. This approach leverages efficient fine-tuning techniques to adapt powerful base models to specific languages.

Quick Start & Requirements

• Install dependencies: pip install -r requirements.txt
• PyTorch with CUDA support is recommended.
• Translation: python translate.py m2m nl ./output_nl --quant4 --batch_size 20
• Combine checkpoints: python combine_checkpoints.py ./output_nl ./hf_dataset_nl
• Fine-tune: python finetune.py tuned_model_nl hf_dataset_nl "You are a generic chatbot that always answers in Dutch."
• Inference: python run_inference.py tuned_model_nl "You are a generic chatbot that always answers in Dutch." "Wat is de hoofdstad van Nederland?"
• See official quick-start for detailed commands.

Highlighted Details

• Supports LLaMa3, LLaMa2, Mistral, and Mixtral models.
• Offers multiple translation models including OPUS, M2M, MADLAD, NLLB, and Seamless.
• Fine-tuning can be done with QLoRA, DPO, and ORPO.
• Benchmarking script (benchmark.py) available to compare translation model performance.
• Pre-trained adapters for various languages and models are available on Hugging Face Hub.

Maintenance & Community

• Actively developed with recent support for LLaMa3.
• Community contributions are encouraged via pull requests.
• Contact info@commandos.ai for funding inquiries.

Licensing & Compatibility

• The repository itself does not specify a license in the README.
• Base models (LLaMa, Mistral) have their own licenses, which may restrict commercial use.
• Translated datasets and fine-tuned models are often hosted on Hugging Face Hub under various licenses (e.g., Apache 2.0 for some adapters).

Limitations & Caveats

• Translation quality can vary significantly between models and languages.
• The translation step can be time-consuming (e.g., ~36 hours for OASST1 to a single language on a free Colab T4).
• Fine-tuning performance depends heavily on the quality and quantity of the translated data.
• Compatibility with other fine-tuning frameworks like Axolotl is mentioned but not detailed.