clewdr  by Xerxes-2

High-performance LLM reverse proxy

Created 1 year ago
1,229 stars

Top 31.3% on SourcePulse

GitHubView on GitHub
1 Expert Loves This Project
Project Summary

ClewdR is a high-performance LLM reverse proxy designed for Claude and Gemini, offering a full-featured React frontend and an efficient, event-driven architecture. It targets users needing to manage and route requests to these large language models, providing significant performance gains and resource efficiency over script-based solutions.

How It Works

ClewdR leverages Rust's Tokio and Axum frameworks for multi-threaded asynchronous processing, enabling it to handle thousands of requests per second with minimal memory footprint (single-digit MB). Its design emphasizes efficiency through an event-driven approach, decoupled logic, and hot reloading capabilities. It features fingerprint-level Chrome simulation for requests, intelligent cookie management with automatic account status classification, and a fine-grained polling mechanism for optimal resource utilization. Response caching is supported via Moka technology.

Quick Start & Requirements

  • Download the pre-compiled binary for your platform.
  • Access the frontend at http://127.0.0.1:8484 and use the console-displayed Web Admin Password for login.
  • Configure proxy addresses, cookies, and keys via the frontend.
  • For third-party applications (e.g., SillyTavern), use the console-provided API access addresses and API Password.
  • Official Wiki: https://github.com/Xerxes-2/clewdr/wiki

Highlighted Details

  • Claims ten times the performance and one-tenth the resource usage of script language implementations.
  • Supports OpenAI compatible modes for both Claude and Gemini.
  • Includes built-in proxy server support, eliminating the need for TUN.
  • Implements stop sequences and image attachment uploads on the proxy side.

Maintenance & Community

  • Primary development by Xerxes-2.
  • Community resources available via GitHub Wiki.

Licensing & Compatibility

  • License details are not explicitly stated in the README.
  • Compatibility for commercial use or closed-source linking is not specified.

Limitations & Caveats

The README does not specify the license, which may impact commercial adoption. Compatibility details for closed-source linking are also absent.

Health Check
Last Commit

1 month ago

Responsiveness

1 day

Pull Requests (30d)
1
Issues (30d)
1
Star History
40 stars in the last 30 days

Explore Similar Projects

Starred by Yaowei Zheng Yaowei Zheng(Author of LLaMA-Factory), Chip Huyen Chip Huyen(Author of "AI Engineering", "Designing Machine Learning Systems"), and
11 more.

GPTCache by zilliztech

0.1%
8k
Semantic cache for LLM queries, integrated with LangChain and LlamaIndex
Created 3 years ago
Updated 1 year ago
Feedback? Help us improve.