Manifest is a Python library designed to simplify prompt programming with foundation models, targeting researchers and developers working with large language models. It provides a unified API for interacting with various models, supports caching for reproducibility and cost savings, and facilitates prompt iteration and experimentation.

How It Works

Manifest abstracts away the complexities of different model APIs by offering a consistent interface for generation, scoring, and embedding tasks. It supports multiple model providers (OpenAI, AI21, Cohere, Together, HuggingFace) and allows seamless switching between them. A key feature is its global caching mechanism, which stores model inputs and outputs using SQLite or Redis, enabling faster iteration, reproducible results, and reduced API costs.

Quick Start & Requirements

• Install: pip install manifest-ml
• Install with diffusion support: pip install manifest-ml[diffusers]
• Install with HuggingFace local model support: pip install manifest-ml[api]
• Requires API keys for cloud providers. Local HuggingFace models require a separate Flask server setup.
• See examples: examples

Highlighted Details

• Unified API for generate, score, and embed tasks.
• Supports caching with SQLite or Redis backends.
• Enables model pooling for parallel and distributed inference.
• Facilitates local inference of HuggingFace models via a Flask API.
• Supports streaming responses and asynchronous batch queries.

Maintenance & Community

• Developed by HazyResearch.
• Roadmap includes clients for HuggingFace Hub, Anthropic, and streaming support for chat models.
• Citation provided for academic use.

Licensing & Compatibility

• License: MIT.
• Compatible with commercial and closed-source applications.

Limitations & Caveats

The model pooling feature is noted as "very much a work in progress." Support for diffusion models is planned but not yet implemented.