This project provides a unified HTTP proxy API for various chat completion and embedding models, assistants, and applications, simplifying integration for developers and users. It's built with Java 17 and Eclipse Vert.x, offering a robust backend for AI-powered services.

How It Works

AI DIAL Core acts as an intelligent gateway, abstracting the complexities of interacting with diverse AI models. It leverages Eclipse Vert.x for asynchronous, non-blocking I/O, enabling high throughput and efficient resource utilization. The core supports dynamic configuration of routes, models, applications, and identity providers, allowing for flexible and scalable deployment. It also integrates with blob storage (S3, Azure Blob, GCS) and Redis for data persistence and caching.

Quick Start & Requirements

• Build: ./gradlew build
• Run: ./gradlew :server:run
• Prerequisites: Java 17, Gradle. Requires GitHub credentials (username and PAT with read:packages permission) for JClouds dependencies, configurable via environment variables (GPR_USERNAME, GPR_PASSWORD) or Gradle properties.
• Deployment: Helm charts are available for Kubernetes deployment.
• Configuration: Extensive configuration options are available via static settings (environment variables, aidial.settings.json) and dynamic settings (JSON files for routes, addons, models, etc.).

Highlighted Details

• Unified API for chat completion and embedding models.
• Extensible through dynamic configuration of routes, models, applications, and addons.
• Supports various storage providers (S3, Azure Blob, GCS) and Redis for caching.
• Includes features like rate limiting, role-based access control, and load balancing.

Maintenance & Community

The project is maintained by EPAM Systems. Further community or roadmap information is not detailed in the README.

Licensing & Compatibility

Licensed under the Apache License, Version 2.0. This license is permissive and allows for commercial use and integration with closed-source projects.

Limitations & Caveats

Building the project requires GitHub credentials due to dependencies on private GitHub packages, which can be an initial hurdle for anonymous users. The configuration options are extensive, requiring careful study for proper setup.