This project provides a free, self-hostable API endpoint for Groq's large language models, targeting developers and researchers who need flexible access to powerful AI without direct Groq cloud costs. It offers a local alternative for integrating LLMs into applications.

How It Works

The project wraps Groq's inference capabilities within a Docker container, exposing a standard OpenAI-compatible API. This allows users to interact with various Groq models (Gemma, Mixtral, Llama) via familiar RESTful requests, abstracting away the complexities of direct Groq SDK integration.

Quick Start & Requirements

• Primary install / run command: docker run -d -p 8080:8080 ghcr.io/star-studio-develop/groq2api:latest
• Prerequisites: Docker.
• Deployment options: Vercel, Koyeb, Render, Railway.
• Usage example: curl -X POST http://127.0.0.1:8080/v1/chat/completions -H 'Authorization: Bearer stytch_session' -d '{"messages": [{"role": "user", "content": "hi"}], "model": "mixtral-8x7b-32768", "max_tokens": 4096, "stream": true}'

Highlighted Details

• Supports multiple Groq models: gemma-7b-it, mixtral-8x7b-32768, llama2-70b-4096, llama3-8b-8192, llama3-70b-8192.
• OpenAI-compatible API endpoint (/v1/chat/completions).
• Configurable parameters: model, stream, max_tokens.
• Offers deployment guides for cloud platforms.

Maintenance & Community

No specific contributors, sponsorships, or community links (Discord/Slack, roadmap) are mentioned in the provided README.

Licensing & Compatibility

The license is not specified in the README. Compatibility for commercial use or closed-source linking is undetermined.

Limitations & Caveats

The README does not specify the license, making commercial use uncertain. There is no information on error handling, rate limiting, or authentication beyond a placeholder Bearer stytch_session token.