Sherlock is an open-source tool designed to intercept and visualize Large Language Model (LLM) API traffic, specifically focusing on token usage. It targets developers working with LLM CLI tools, offering a real-time terminal dashboard to monitor token consumption, track costs, debug prompts, and manage context window utilization, thereby enhancing development efficiency and cost awareness.

How It Works

Sherlock operates by setting up an HTTP proxy that intercepts requests made by supported LLM CLI tools. This intercepted traffic is then processed to display real-time token usage in a live terminal dashboard, featuring a visual "fuel gauge" indicating cumulative usage against a user-defined limit. Additionally, it automatically archives every prompt and its associated metadata in both human-readable Markdown and raw JSON formats, facilitating detailed review and debugging. The project emphasizes a zero-configuration approach, aiming for seamless integration into existing development workflows.

Quick Start & Requirements

• Installation: Clone the repository (git clone https://github.com/jmuncor/sherlock.git), navigate into the directory (cd sherlock), and install using pip install -e ..
• Prerequisites: Python 3.10 or newer.
• Running: Start the dashboard with sherlock start. Run LLM tools with proxy configuration using commands like sherlock claude or sherlock codex.
• Links: Repository: https://github.com/jmuncor/sherlock

Highlighted Details

• Live Terminal Dashboard: Provides real-time token tracking with color-coded context window usage (Green: < 50%, Yellow: 50-80%, Red: > 80% of limit).
• Prompt Archive: Automatically saves all intercepted requests as Markdown files and JSON objects for review and debugging.
• Session Summary: Displays total token usage across all requests upon exiting the session.
• Zero Configuration: Designed for immediate use without complex setup or certificate management.

Maintenance & Community

No specific details regarding maintainers, community channels (like Discord or Slack), or a public roadmap are provided in the README.

Licensing & Compatibility

This project is licensed under the MIT License, which is generally permissive for commercial use and integration into closed-source projects.

Limitations & Caveats

The integration for Google's Gemini CLI is currently non-functional due to an upstream issue within the Gemini CLI itself, which prevents it from respecting custom base URLs when using OAuth authentication. Support for Gemini will be restored once this issue is resolved by the Gemini CLI team.