This repository provides a curated collection of Jinja-formatted chat templates and corresponding generation configuration files for various instruction-tuned Large Language Models (LLMs) compatible with the Hugging Face transformers library. It aims to simplify the correct formatting of conversational inputs for these models, enabling users to leverage their chat capabilities effectively.

How It Works

The project centralizes Jinja templates that define the specific prompt structure required by different LLMs for conversational interactions. These templates are designed to be loaded directly into the tokenizer.chat_template attribute, ensuring that user messages, system prompts, and assistant responses are formatted according to the model's training data. Accompanying JSON files provide generation configurations, including stop_token_ids, to further refine model output.

Quick Start & Requirements

• Install: No explicit installation required; templates are loaded directly from the repository's files.
• Prerequisites: Hugging Face transformers library.
• Usage: Load a template file (e.g., open('./chat_templates/llama-3-instruct.jinja').read()) and assign it to tokenizer.chat_template.
• Docs: Usage Examples

Highlighted Details

• Supports a wide range of popular LLMs including Llama, Mistral, Gemma, Phi-3, Qwen, and more.
• Includes specific notes on handling system messages for models like Mistral and Gemma.
• Provides generation configuration files for fine-tuning response termination.
• Regularly updated with support for new model releases.

Maintenance & Community

The repository is actively updated, with recent additions including support for Llama-3.2, Granite-3.0, and Phi-3 models. The author notes potential for reduced active maintenance due to Hugging Face's increasing integration of chat templates within tokenizer_config.json. Contributions via pull requests are welcomed.

Licensing & Compatibility

The repository does not explicitly state a license. The code snippets provided are for demonstration and can be used freely. Compatibility with commercial or closed-source projects depends on the licenses of the underlying LLMs and the Hugging Face transformers library.

Limitations & Caveats

The author notes that they may not actively maintain the repository, as Hugging Face models are increasingly embedding chat templates directly into tokenizer_config.json. Some models, like older Vicuna versions, may have specific quirks or perform better with certain template variations.