Unified API for reranking models, simplifying retrieval architectures
Top 28.0% on sourcepulse
This library provides a unified, lightweight API for various reranking models, simplifying integration into retrieval pipelines. It targets developers and researchers working with semantic search and information retrieval systems, offering a consistent interface to diverse reranking architectures, including cross-encoders, LLM-based methods, and API providers.
How It Works
The core of the library is a single Reranker
class that abstracts away the complexities of different reranking models. It supports a wide array of architectures, from standard Hugging Face cross-encoders and T5 models to API-based services like Cohere and Pinecone, and specialized implementations like FlashRank and RankLLM. The library aims for minimal dependencies by default, allowing users to install only the necessary components for the models they intend to use.
Quick Start & Requirements
pip install rerankers
pip install "rerankers[transformers]"
, pip install "rerankers[gpt]"
, pip install "rerankers[api]"
, pip install "rerankers[flashrank]"
, pip install "rerankers[rankllm]"
, pip install "rerankers[monovlm]"
, pip install "rerankers[llmlayerwise]"
, or pip install "rerankers[all]"
.Highlighted Details
Maintenance & Community
Developed by Benjamin Clavié as part of Answer.ai. Recent updates include support for MXBai V2, removal of Pydantic/tqdm dependencies, Pinecone API rerankers, and multi-modal support.
Licensing & Compatibility
The project is available under the MIT License, permitting commercial use and integration with closed-source projects.
Limitations & Caveats
Training functionality is not directly provided by the library, though it aims to interface with libraries that support training. RankZephyr/RankVicuna support via RankLLM is untested. Reproducibility for RankGPT can be challenging.
2 months ago
1 week