ATLAS provides a principled benchmark and dataset for formulating effective prompts for large language models (LLMs). It introduces 26 guiding principles to optimize LLM interactions, benefiting researchers and practitioners aiming to improve LLM query design and comprehension.

How It Works

The project leverages a curated dataset of 13,000 data points, categorized by 26 distinct prompting principles. These principles are designed to enhance LLM performance across various scales, from LLaMA to GPT-4. The dataset includes both a general collection and individual principle-specific files, facilitating focused analysis and fine-tuning.

Quick Start & Requirements

• The dataset is available as general_dataset.json and individual principle files.
• Compatibility is noted with Stanford Alpaca and FastChat for principled instruction fine-tuning.
• Links to related tools like Prompt Enhancer, Magic Prompts, and Prompt-builder are provided.

Highlighted Details

• Introduces 26 guiding principles for LLM prompt formulation.
• Dataset comprises 13,000 data points, including model-generated responses.
• Principles validated through experiments on LLaMA-1/2 and GPT-3.5/4.
• Compatible with Stanford Alpaca and FastChat for fine-tuning.

Maintenance & Community

The project acknowledges contributions from Lim Hyo Jeong, Lyzr, and lypsoty112 for associated tools. Further contributions to principles and the dataset are welcomed.

Licensing & Compatibility

The README does not explicitly state the license type or compatibility for commercial use.

Limitations & Caveats

The repository focuses on prompt formulation principles and provides a dataset; it does not directly offer pre-trained or fine-tuned models, though it mentions plans to release them.