token-optimizer-mcp  by ooples

Intelligent token optimization for LLM context windows

Created 7 months ago
395 stars

Top 72.7% on SourcePulse

GitHubView on GitHub
Project Summary

Summary

This project addresses the challenge of excessive context window usage in AI models like Claude by providing intelligent token optimization. It targets users of Claude Code, Claude Desktop, Cursor IDE, and other integrated AI tools, offering significant token reduction (60-90%) through caching, compression, and smart tool replacements. The primary benefit is maximizing available context window and reducing operational costs.

How It Works

The Token Optimizer MCP operates as a Model Context Protocol (MCP) server. Its core approach involves storing compressed content externally in an SQLite database, thereby freeing up the AI's context window. It employs Brotli compression for efficient data reduction (2-4x typical, up to 82x for repetitive data) and utilizes tiktoken for accurate token counting. The system also provides optimized replacements for common tools like file operations (read, grep, glob), API calls, and database queries, often using diff-based updates or caching for substantial token savings.

Quick Start & Requirements

Installation is performed globally via npm: npm install -g @ooples/token-optimizer-mcp. The project requires Node.js 20+ and automatically detects and configures supported AI tools (Claude Code, Claude Desktop, Cursor IDE, Cline, etc.) through global hooks upon installation. Detailed platform-specific installation instructions are available in docs/HOOKS-INSTALLATION.md.

Highlighted Details

  • Achieves 60-90% token reduction across 38,000+ real-world operations.
  • Brotli compression offers typical 2-4x reduction, scaling up to 82x for highly repetitive content.
  • Provides over 60 specialized tools covering file operations, API/database management, build/testing, advanced caching, and monitoring.
  • Reports high cache hit rates: >80% for file operations, >75% for API responses, and >70% for database queries.

Maintenance & Community

The project is developed by the "ooples team". The README does not specify details regarding community channels (e.g., Discord, Slack), active contributors, or sponsorships.

Licensing & Compatibility

Released under the MIT License, this project offers broad compatibility for commercial use and integration without copyleft restrictions.

Limitations & Caveats

Optimization is most effective for text content exceeding 500 characters. Cache entries are automatically cleaned up after 7 days to manage disk usage. Token counting uses the GPT-4 tokenizer, which serves as an approximation for Claude's tokenizer.

Health Check
Last Commit

3 days ago

Responsiveness

Inactive

Pull Requests (30d)
3
Issues (30d)
0
Star History
102 stars in the last 30 days

Explore Similar Projects

Feedback? Help us improve.