SparseML is an open-source toolkit for optimizing neural networks through pruning, quantization, and distillation, targeting researchers and engineers who want to create faster, smaller models for efficient inference. It enables applying state-of-the-art sparsification techniques with minimal code changes, exporting models to ONNX for deployment with DeepSparse on CPU hardware.

How It Works

SparseML utilizes a declarative recipe system (YAML files) to define sparsification algorithms and hyperparameters. These recipes are parsed and applied to PyTorch or Hugging Face models via a Python API or a CLI. The library abstracts the complexity of these algorithms, allowing users to integrate sparsification into existing training pipelines with minimal code modification.

Quick Start & Requirements

• Install with pip: pip install sparseml
• Tested on Python 3.8-3.11, Linux/Debian.
• Supported frameworks: PyTorch (>=1.1.0, <=2.0), TensorFlow (>=1.8.0, <2.0.0), TensorFlow.Keras (>=2.2.0).
• For Hugging Face Transformers integration: pip install -e "sparseml[transformers]"
• Documentation: SparseML Docs

Highlighted Details

• Supports Sparse Transfer Learning from SparseZoo (pre-sparsified models) and Sparsification from Scratch for arbitrary models.
• Offers one-shot LLM compression workflows with the new SparseGPTModifier.
• Integrates with PyTorch, Hugging Face Transformers, and Ultralytics YOLOv5/v8.
• Models can be exported to ONNX for deployment with DeepSparse.

Maintenance & Community

• Active development by Neural Magic.
• Community support via Slack: Neural Magic Community Slack
• GitHub Issues for bugs and feature requests.

Licensing & Compatibility

• Licensed under the Apache License Version 2.0.
• Permissive license suitable for commercial use and integration with closed-source projects.

Limitations & Caveats

The library supports specific versions of PyTorch and TensorFlow; compatibility with newer versions may require updates. TensorFlow support is limited to versions < 2.0.0.