Tensor library for machine learning
Top 3.9% on sourcepulse
ggml is a C tensor library designed for machine learning, focusing on efficient execution across diverse hardware. It targets developers and researchers needing a low-level, dependency-free tensor computation engine for inference and training, particularly on resource-constrained devices. The library's key benefit is its portability and performance through features like integer quantization and broad hardware acceleration.
How It Works
ggml employs a C-based, low-level implementation for maximum portability and minimal overhead. It supports integer quantization to reduce model size and memory bandwidth, enabling faster inference on CPUs and GPUs. The library handles automatic differentiation and includes optimizers like ADAM and L-BFGS, facilitating both inference and training workflows. A notable design choice is its commitment to zero memory allocations during runtime, contributing to predictable performance.
Quick Start & Requirements
pip install -r requirements.txt
.mkdir build && cd build && cmake .. && cmake --build . --config Release -j 8
.Highlighted Details
Maintenance & Community
Development is active, with significant contributions seen in related projects like llama.cpp
and whisper.cpp
.
Licensing & Compatibility
The repository does not explicitly state a license in the provided README. Users should verify licensing for commercial or closed-source use.
Limitations & Caveats
The project is under active development, implying potential for breaking changes. Specific hardware acceleration configurations (CUDA, hipBLAS, SYCL) require careful setup and may have version dependencies.
3 days ago
1 day