Sagify simplifies the deployment and management of Large Language Models (LLMs) and machine learning workflows on AWS SageMaker. It offers a unified interface, the LLM Gateway, to interact with both proprietary (OpenAI) and open-source models for tasks like chat completions, image generation, and embeddings, abstracting away infrastructure complexities for ML engineers and researchers.

How It Works

Sagify utilizes a modular architecture centered around an LLM Gateway, a FastAPI application that provides a consistent API for various LLM providers. It supports deploying open-source models directly to AWS SageMaker endpoints, allowing for fine-grained control over instance types and scaling. Alternatively, it can interface with OpenAI's API for proprietary models. The gateway acts as a central hub, routing requests to the appropriate backend service, whether it's a SageMaker endpoint or the OpenAI API.

Quick Start & Requirements

• Installation: pip install sagify
• Prerequisites: Python (3.7-3.11), Docker, configured AWS CLI.
• Deployment Example: sh sagify cloud foundation-model-deploy --model-id model-txt2img-stabilityai-stable-diffusion-v2-1-base --model-version 1.* -n 1 -e ml.p3.2xlarge --aws-region us-east-1 --aws-profile sagemaker-dev
• LLM Gateway: Can be run locally via Docker (sagify llm gateway --image sagify-llm-gateway:v0.1.0 --start-local) or deployed to AWS Fargate.
• Documentation: Read the Docs

Highlighted Details

• Supports a wide range of open-source models (Llama-2, Stable Diffusion, Sentence Transformers) and OpenAI models (GPT-4, DALL-E 3).
• Enables deployment of open-source models to SageMaker endpoints with configurable instance types and counts.
• Provides a unified API for chat completions, image generation, and embeddings.
• Offers deployment options for the LLM Gateway on local Docker or AWS Fargate.

Maintenance & Community

• Contact: Email: pavlos@sagify.ai

Licensing & Compatibility

• The README does not explicitly state the license.

Limitations & Caveats

• The project is described as simplifying ML workflows, but the setup and configuration for deploying models to SageMaker and running the LLM Gateway require significant AWS and Docker knowledge.
• The README does not specify the license, which could be a blocker for commercial use or integration into closed-source projects.