claude-code-cache-fix  by cnighswonger

Claude Code cache fix proxy and module

Created 3 months ago
358 stars

Top 77.9% on SourcePulse

GitHubView on GitHub
Project Summary

This project addresses significant cost increases and performance regressions in Anthropic's Claude Code, particularly on resumed sessions, by acting as a transparent HTTP proxy or Node.js interceptor. It targets power users and developers seeking to optimize API usage and maintain stable, cost-effective interactions with Claude Code. The primary benefit is a substantial reduction in API costs through improved prompt cache hit rates and prevention of silent quota burn.

How It Works

The core approach involves a local HTTP proxy (or a Node.js --import module for older versions) that sits between Claude Code and the Anthropic API. This proxy applies a pipeline of hot-reloadable extensions to requests and responses. Key extensions normalize request structures by stabilizing tool ordering, stripping unstable version fingerprints, managing Time-To-Live (TTL) markers, and normalizing identity fields. This mechanism directly combats cache-busting bugs without altering Claude Code's core functionality, offering enhanced observability into session costs and behavior.

Quick Start & Requirements

  • Primary Install/Run (Proxy):
    1. Install: npm install -g claude-code-cache-fix
    2. Start Proxy: node "$(npm root -g)/claude-code-cache-fix/proxy/server.mjs" &
    3. Launch Claude Code: ANTHROPIC_BASE_URL=http://127.0.0.1:9801 claude
  • Docker: docker run -d --name cache-fix-proxy --restart=always -p 9801:9801 ghcr.io/cnighswonger/claude-code-cache-fix:latest
  • Prerequisites: Node.js or Bun runtime. Docker for containerized deployment.
  • Links: Proxy Test Harness, Disclosure Record, Monitoring Docs.

Highlighted Details

  • Performance: A/B tests show a 95.5% cache hit rate via the proxy compared to 82.3% direct on the first warm turn.
  • Cost Optimization: Fixes bugs causing up to 20x cost increases on resumed sessions and addresses Opus 4.7's higher token burn rate.
  • Bootstrap Channel Defense: Secures prompt injection surfaces (/api/claude_cli/bootstrap, GrowthBook flags) with audit, block, and allowlist modes.
  • Image Handling: Features like CACHE_FIX_IMAGE_KEEP_LAST and CACHE_FIX_IMAGE_MAX_DIM mitigate costs associated with image data in conversations.
  • Observability: Provides real-time status lines and detailed telemetry in ~/.claude/quota-status/ for monitoring quota, TTL, and cache performance.
  • Session Health: Offers early warnings for thinking-desync risks in long Opus 4.7 sessions.

Maintenance & Community

The project lists numerous contributors, including key figures like @cnighswonger, @ArkNill, @bilby91, and @fgrosswig, indicating active community involvement. Production usage is noted by Crunchloop DAP and VM Farms. Community interaction primarily occurs through GitHub issues and discussions.

Licensing & Compatibility

  • License: MIT License.
  • Compatibility: Works with various Claude Code versions, including v2.1.113+ Bun binary. Supports corporate proxy environments via standard environment variables (HTTPS_PROXY, NO_PROXY, CACHE_FIX_PROXY_CA_FILE).

Limitations & Caveats

The proxy requires a running process before Claude Code initialization; running it as a service is recommended. It cannot prevent server-side TTL downgrades triggered by quota overage, though it aims to reduce the likelihood of overage. Microcompact degradation is monitored but not preventable. The system prompt rewrite feature is experimental and preload-only. Major Claude Code refactors may necessitate package updates.

Health Check
Last Commit

5 days ago

Responsiveness

Inactive

Pull Requests (30d)
20
Issues (30d)
10
Star History
72 stars in the last 30 days

Explore Similar Projects

Starred by Yaowei Zheng Yaowei Zheng(Author of LLaMA-Factory), Chip Huyen Chip Huyen(Author of "AI Engineering", "Designing Machine Learning Systems"), and
11 more.

GPTCache by zilliztech

0.1%
8k
Semantic cache for LLM queries, integrated with LangChain and LlamaIndex
Created 3 years ago
Updated 1 year ago
Feedback? Help us improve.