REST API gateway for Amazon Bedrock
Top 45.8% on sourcepulse
This project provides OpenAI-compatible RESTful APIs for Amazon Bedrock, enabling developers to leverage Amazon's foundation models (like Claude 3, Llama 3) using existing OpenAI SDKs and tools without code modifications. It targets developers and researchers seeking to experiment with or integrate Bedrock models into applications originally built for OpenAI.
How It Works
The gateway acts as a proxy, translating OpenAI API requests into Amazon Bedrock API calls. It supports various Bedrock capabilities including chat completions, streaming responses via Server-Sent Events (SSE), tool calls, embeddings, and multimodal inputs. The architecture can be deployed using AWS Lambda or AWS Fargate behind an Application Load Balancer (ALB), with options for Lambda Function URLs for potentially lower latency.
Quick Start & Requirements
Highlighted Details
Maintenance & Community
The project is part of the aws-samples
organization, indicating official AWS backing. No specific community links (Discord/Slack) are provided in the README.
Licensing & Compatibility
Limitations & Caveats
Currently does not support SageMaker models or Bedrock custom/fine-tuned models with Provisioned Throughput. Performance may incur additional latency compared to direct AWS SDK calls.
1 week ago
1 week