This project provides an OpenAI-compatible API layer for Cloudflare's Workers AI platform, enabling developers to leverage smaller, cost-effective LLMs with existing OpenAI SDKs. It targets developers seeking to migrate from OpenAI's proprietary models to alternative, self-hosted solutions without extensive code refactoring.

How It Works

The project acts as a proxy, translating OpenAI API requests into calls to Cloudflare Workers AI endpoints. It supports various OpenAI functionalities like completions, chat completions, audio transcription, embeddings, and image generation using models like Llama 2 and Whisper. This approach allows for seamless integration with existing applications that rely on the OpenAI API structure.

Quick Start & Requirements

• Install dependencies: yarn
• Deploy: yarn deploy
• Set secrets: wrangler secret put ACCESS_TOKEN, wrangler secret put CLOUDFLARE_API_TOKEN
• Requires Cloudflare Account ID, Cloudflare API token with Workers AI template, and Node.js/Yarn.
• Local testing is not supported; remote testing via yarn dev proxies to deployed API.
• Official docs: https://github.com/chand1012/openai-cf-workers-ai

Highlighted Details

• Replaces OpenAI API calls with Cloudflare Workers AI.
• Supports Completions, Chat Completions, Audio Transcription, Embeddings, Image Generation.
• Uses models like Llama 2 for chat and Whisper for transcription.
• Aims to provide a cost-effective alternative to OpenAI's API.

Maintenance & Community

The project is maintained by chand1012. Updates are described as semi-regular, depending on Workers AI feature rollouts. No specific community channels (Discord/Slack) are mentioned.

Licensing & Compatibility

Licensed under the MIT License. This license permits commercial use and integration with closed-source projects.

Limitations & Caveats

The API does not count tokens, always returning zero for the usage attribute. Stop tokens are non-functional and may sometimes appear in responses. Features like Files, Assistants, Threads, and Runs are planned but not yet implemented. Image editing and variants are not supported.