Model Manager is a Python package designed to streamline the deployment of open-source AI models to AWS, targeting developers and researchers who want to quickly set up inference endpoints without extensive cloud configuration. It simplifies the process of launching SageMaker instances for models sourced from Hugging Face or SageMaker itself, providing a ready-to-query API endpoint in minutes.

How It Works

The tool leverages AWS SageMaker to provision and manage inference infrastructure. Users select a model from Hugging Face or SageMaker, and Model Manager automates the creation of a SageMaker endpoint. It supports deploying models via direct selection, YAML configuration files for reproducibility, and handles model uploads from local paths. Deployed models can be queried via a generated FastAPI server or directly through the SageMaker API.

Quick Start & Requirements

• Install: Clone the repository and run bash setup.sh.
• Prerequisites: Python, an AWS account with SageMaker instance quota, and potentially a Hugging Face Hub token for restricted models.
• Setup: The setup.sh script configures the AWS client, prompting for AWS credentials and region. Hugging Face token can be added to a .env file.
• Docs: Getting Started

Highlighted Details

• Deploys models from Hugging Face or SageMaker to AWS SageMaker endpoints.
• Supports deployment via CLI commands or reproducible YAML configuration files.
• Offers a FastAPI server for easy querying or direct SageMaker API integration.
• Handles model deployment from Hugging Face, SageMaker, or custom S3/local paths.

Maintenance & Community

The project is maintained by Arthur & Tyler, reachable at hello@openfoundry.ai. Future work includes Azure/GCP support, enhanced error handling, and autoscaling.

Licensing & Compatibility

Distributed under the MIT License, allowing for commercial use and integration with closed-source applications.

Limitations & Caveats

Currently, querying within Model Manager is limited to text-based models; multimodal and image generation models are not yet supported. Model versions are static, and deleting endpoints may not be instantaneous. Deploying identical models within the same minute can cause issues.