This repository provides a curated and regularly updated comparison of open-source local LLM inference projects. It targets developers and researchers evaluating LLM tooling by offering a quantitative overview of project popularity and maintenance activity based on GitHub metrics.

How It Works

The project aggregates GitHub repository data (stars, forks, issues, contributors, releases, last commit time) for various LLM inference backends, frontends, and all-in-one applications. This data is presented in a sortable table, filtered to include only active and reasonably popular projects (over 100 stars, committed within the last 60 days). The methodology uses these metrics as a proxy for community adoption and ongoing development.

Quick Start & Requirements

This repository is a curated list and does not require installation or execution. It links to a Google Sheet for a more comprehensive dataset and provides a blog post with qualitative analysis of local LLM tooling.

Highlighted Details

• Comprehensive metrics for over 50 local LLM projects.
• Filters applied to highlight active and popular repositories.
• Links to a detailed Google Sheet and author's analysis.
• Community contributions are encouraged to expand the list.

Maintenance & Community

The list is updated regularly, with the last update noted as 27/01/2025. Contributions are welcomed via GitHub Issues or pull requests.

Licensing & Compatibility

The repository itself does not have a license specified, but it lists the licenses of the projects it tracks, which vary widely from MIT and Apache 2.0 to GPL and AGPL.

Limitations & Caveats

The metrics used (stars, forks, etc.) are proxies and may not perfectly reflect a project's technical merit or suitability for all use cases. The filtering criteria (100+ stars, commit within 60 days) exclude smaller or less active projects.