This repository provides Tamil-Llama, a family of LLaMA-based Large Language Models specifically adapted for the Tamil language. It offers instruction-following and base models in 7B and 13B parameter sizes, aiming to match or exceed LLaMA 2 performance on various benchmarks, including English reasoning tasks.

How It Works

Tamil-Llama builds upon Meta's LLaMA models by incorporating additional Tamil tokens and utilizing LoRA for efficient fine-tuning. The project also includes pre-training on Tamil Wikipedia data and fine-tuning on a mix of Tamil Alpaca and English Alpaca datasets. Recent updates include experimental adapters for Google's Gemma 2B model.

Quick Start & Requirements

• LM Studio: Download LM Studio, search for "Tamil Llama", download a variant (e.g., GGUF Q8_0), and import the model_config.json from the repo.
• Ollama: Ensure Ollama is installed. Download the Modelfile and a GGUF model file (e.g., tamil-llama.gguf). Place them in the same directory and run ollama create tamil-llama -f Modelfile.
• Dependencies: Requires models to be downloaded. LM Studio supports Windows, macOS (Silicon), and Linux (beta). Ollama is cross-platform.
• Resources: GGUF models are quantized for efficiency. Specific hardware requirements depend on the chosen model size and quantization level.
• Demos: Google Colab notebook and HuggingFace Space demo available.

Highlighted Details

• Models match or outperform LLaMA 2 on most benchmarks, including English reasoning.
• Tamil LLaMA 7B Instruct ranked top on the Open LLM Leaderboard for models under 3B parameters (as of Feb 2023).
• Includes Tamil-translated Alpaca and OpenOrca datasets for fine-tuning.
• Experimental adapters for Google Gemma 2B are available.

Maintenance & Community

The project is maintained by Abhinand Balachandran. Contributions are welcomed via issues or pull requests. Contact: abhinandb.ml@gmail.com.

Licensing & Compatibility

The source code and datasets are licensed under GNU GPL v3.0. However, the models are derivatives of LLaMA 2 and are subject to its original licensing terms. Users must consult the LLAMA2-LICENSE file for comprehensive licensing details, which may restrict commercial use or linking with closed-source applications.

Limitations & Caveats

The models have not undergone detoxification and may generate harmful or offensive content. Users are urged to exercise discretion and supervise outputs. The Gemma 2B adapter is an experimental release.