This repository serves as a comprehensive catalog and guide for Large Language Models (LLMs), targeting researchers, developers, and enthusiasts interested in the rapidly evolving LLM landscape. It provides detailed information on pre-trained and fine-tuned models, training techniques, datasets, and evaluation benchmarks, aiming to demystify LLM development and application.

How It Works

The project functions as a curated knowledge base, aggregating information from various sources to present a structured overview of LLMs. It categorizes models by developer (e.g., OpenAI, Meta, Google), highlights key architectural choices (e.g., Transformer, RNN with linear attention), and details training methodologies, including parameter-efficient fine-tuning (PEFT) techniques like LoRA and QLoRA. The repository also lists and describes numerous datasets used for pre-training and fine-tuning, alongside a suite of evaluation benchmarks for assessing LLM performance.

Quick Start & Requirements

This repository is primarily informational and does not require installation or execution. It links to numerous external projects and resources for practical LLM implementation.

Highlighted Details

• Extensive catalog of over 100 pre-trained and fine-tuned LLMs from major research labs and companies.
• Detailed explanations of efficient training techniques (PEFT, quantization) and parallelization strategies (FSDP, ZeRO).
• Comprehensive list of instruction and pre-training datasets, including Hugging Face, OpenAssistant, and custom collections.
• Overview of LLM evaluation benchmarks like MMLU, HELM, and BIG-Bench, with links to leaderboards.

Maintenance & Community

The repository appears to be a community-driven effort, with numerous links to related GitHub projects and resources, indicating active engagement within the LLM research community. Specific maintainer information or community channels are not explicitly detailed.

Licensing & Compatibility

The repository itself does not host code but links to projects with various licenses, including Apache 2.0, MIT, CC-BY-SA-4.0, and CC-BY-NC-SA-4.0. Users must consult the licenses of individual linked projects for compatibility, especially for commercial use. Some models, like LLaMA, were initially released under non-commercial licenses.

Limitations & Caveats

The repository is a static collection of information and does not provide executable code or direct access to models. Information may become outdated as the LLM field progresses rapidly. Some linked projects may have specific hardware or software requirements not detailed here.