This repository provides a curated list of Large Language Models (LLMs) that can be deployed on consumer hardware, along with direct links to Google Colab notebooks for easy testing. It aims to help users compare and evaluate LLMs for their specific use cases, moving beyond simple performance scores to direct hands-on experience.

How It Works

The project focuses on providing accessible, one-click testing environments for various LLMs. Instead of relying solely on benchmark scores, it offers Colab links that allow users to interact with different models directly. This approach emphasizes the subjective nature of LLM performance, where suitability often depends on the user's specific needs and preferences.

Quick Start & Requirements

• To use: Click the "Open in Colab" button associated with each model.
• Requirements: A Google account is required to use Google Colab. Some models may have specific hardware requirements (e.g., GPU) which are managed by the Colab environment.
• Resources: Colab provides free GPU access, but usage limits may apply.

Highlighted Details

• Extensive list of GGUF-quantized models, covering a wide range of sizes and capabilities.
• Categorization of models by their suitability for tasks like roleplaying, coding, or NSFW content.
• Regular updates with new models and links, reflecting the rapid pace of LLM development.
• Includes both older, scored models and newer models for direct user evaluation.

Maintenance & Community

The project is actively maintained, with frequent updates to the model list. It draws heavily on community suggestions and contributions, particularly from platforms like Reddit (r/LocalLLaMA). The project acknowledges and credits key open-source projects and individuals that enable this work, such as gpt4all, llama.cpp, oobabooga/text-generation-webui, and "The Bloke" for model quantizations.

Licensing & Compatibility

The repository itself does not specify a license. The LLMs linked are typically hosted on Hugging Face and are subject to their respective licenses, which vary. Users should verify the license of each model before commercial use.

Limitations & Caveats

The project is marked as "WIP" (Work In Progress), indicating that it is under active development and may undergo changes. Some older GGML models might not be compatible with the latest versions of llama.cpp due to breaking changes. The "scores" for older models are based on specific GPT-4 versions and may not be directly comparable to current benchmarks.