clewdr  by Xerxes-2

High-performance LLM reverse proxy

created 4 months ago
425 stars

Top 70.5% on sourcepulse

GitHubView on GitHub
Project Summary

ClewdR is a high-performance LLM reverse proxy designed for Claude and Gemini, offering a full-featured React frontend and an efficient, event-driven architecture. It targets users needing to manage and route requests to these large language models, providing significant performance gains and resource efficiency over script-based solutions.

How It Works

ClewdR leverages Rust's Tokio and Axum frameworks for multi-threaded asynchronous processing, enabling it to handle thousands of requests per second with minimal memory footprint (single-digit MB). Its design emphasizes efficiency through an event-driven approach, decoupled logic, and hot reloading capabilities. It features fingerprint-level Chrome simulation for requests, intelligent cookie management with automatic account status classification, and a fine-grained polling mechanism for optimal resource utilization. Response caching is supported via Moka technology.

Quick Start & Requirements

  • Download the pre-compiled binary for your platform.
  • Access the frontend at http://127.0.0.1:8484 and use the console-displayed Web Admin Password for login.
  • Configure proxy addresses, cookies, and keys via the frontend.
  • For third-party applications (e.g., SillyTavern), use the console-provided API access addresses and API Password.
  • Official Wiki: https://github.com/Xerxes-2/clewdr/wiki

Highlighted Details

  • Claims ten times the performance and one-tenth the resource usage of script language implementations.
  • Supports OpenAI compatible modes for both Claude and Gemini.
  • Includes built-in proxy server support, eliminating the need for TUN.
  • Implements stop sequences and image attachment uploads on the proxy side.

Maintenance & Community

  • Primary development by Xerxes-2.
  • Community resources available via GitHub Wiki.

Licensing & Compatibility

  • License details are not explicitly stated in the README.
  • Compatibility for commercial use or closed-source linking is not specified.

Limitations & Caveats

The README does not specify the license, which may impact commercial adoption. Compatibility details for closed-source linking are also absent.

Health Check
Last commit

2 days ago

Responsiveness

Inactive

Pull Requests (30d)
4
Issues (30d)
6
Star History
338 stars in the last 90 days

Explore Similar Projects

Feedback? Help us improve.