This C library and CLI toolset provides utilities for manipulating GGUF files, a format central to local machine learning inference. It's designed for developers and researchers working with models like those from llama.cpp, offering deep inspection and comparison capabilities for GGUF files.

How It Works

The project implements the GGUF file format specification in C, providing a foundational library (gguf-tools.c) for parsing and manipulating these files. The CLI tool leverages this library to offer specific functionalities like detailed file inspection, tensor comparison, and tensor value examination. This C-based approach aims for efficiency and accessibility, serving as both a practical tool and a documented reference implementation of the GGUF format.

Quick Start & Requirements

• Install/run: Compile from source using a C compiler (e.g., gcc).
• Prerequisites: C compiler, standard C libraries. No specific Python or GPU requirements are mentioned for the core tools.
• Links: Official GGUF Specification

Highlighted Details

• gguf-tools show: Provides detailed key-value metadata and tensor information, including offsets and dimensions.
• gguf-tools compare: Calculates average weight differences between tensors of two GGUF files to assess model relationships and fine-tuning.
• gguf-tools inspect-tensor: Displays raw weight values for specific tensors, useful for quantization analysis.
• gguf-tools split-mixtral: An experimental tool to extract specific expert models from Mixtral MoE GGUF files.

Maintenance & Community

The project is maintained by "antirez," a notable figure in the C and systems programming community. Documentation is primarily within the source code itself, indicating active development.

Licensing & Compatibility

The repository does not explicitly state a license in the README. Users should verify licensing for commercial or closed-source use.

Limitations & Caveats

The library is described as a "work in progress" and explicitly states that "Many quantization formats are missing." The split-mixtral tool is experimental and may be removed. Documentation beyond the source code is limited.