MAPIE - Model Agnostic Prediction Interval Estimator

MAPIE is a scikit-learn-compatible Python library designed for estimating prediction intervals and controlling risks in machine learning models. It targets researchers and practitioners needing robust uncertainty quantification and probabilistic guarantees across regression, classification, and time series tasks, offering a model-agnostic approach.

How It Works

The library leverages Conformal Prediction and Distribution-Free Inference principles, implementing peer-reviewed, model-agnostic algorithms. It estimates uncertainty by utilizing a dedicated conformalization dataset, providing theoretical guarantees under minimal data and model assumptions. This approach ensures valid coverage and risk control without requiring strong distributional assumptions.

Quick Start & Requirements

MAPIE requires Python >=3.9, NumPy >=1.23, and scikit-learn >=1.4. Installation is straightforward via pip (pip install mapie), conda (conda install -c conda-forge mapie), or directly from GitHub. Comprehensive documentation and quickstart examples for regression and classification tasks are available online.

Highlighted Details

• Provides prediction intervals/sets for regression, classification, and time series.
• Enables risk control for complex tasks like multi-label classification and semantic segmentation, offering probabilistic guarantees on metrics such as recall and precision.
• Features scikit-learn-compatible wrappers, allowing integration with models from TensorFlow, PyTorch, and other frameworks.
• MAPIE v1 introduces significant API changes, with a roadmap for 2026 including LLM-as-Judge applications, image segmentation, and enhanced adaptability features.

Maintenance & Community

Developed through a collaboration involving Capgemini Invent, Quantmetry, Michelin, ENS Paris-Saclay, and supported by Région Ile de France and Confiance.ai. Contributions are welcomed via GitHub issues and discussions, with detailed contribution guidelines provided.

Licensing & Compatibility

MAPIE is distributed under the permissive BSD-3-Clause license, allowing for broad compatibility with commercial and closed-source applications.

Limitations & Caveats

Version 1.0 represents a significant API overhaul, potentially introducing breaking changes for existing users. The library's applicability relies on certain data assumptions, and future updates aim to introduce explicit tests to help users verify these conditions.