tokenjuice  by vincentkoc

Token compaction CLI for agent workflows

Created 2 months ago
488 stars

Top 62.4% on SourcePulse

GitHubView on GitHub
1 Expert Loves This Project
Project Summary

Tokenjuice addresses the problem of excessive, noisy terminal output generated by agent workflows, which wastes context and necessitates reruns. It offers a deterministic output compaction solution for developers and users of terminal-heavy agent and coding frameworks, aiming to reduce transcript waste and improve tool handoffs.

How It Works

Tokenjuice functions as a deterministic output compactor. It observes the output of terminal commands executed by agents, applying rule-driven reducers to generate a smaller, semantically equivalent payload. This approach preserves command semantics, avoids making the shell "magical," and provides inspectable JSON rules for clarity. Host integrations act as thin wrappers, leveraging the core reducer logic rather than implementing custom adapter logic.

Quick Start & Requirements

  • Installation: Install globally via package managers (npm install -g tokenjuice, pnpm add -g tokenjuice, yarn global add tokenjuice) or using Homebrew (brew install tokenjuice).
  • Integrations: Install specific host integrations using tokenjuice install [client_name]. Note: OpenClaw integration requires OpenClaw 2026.4.22 or newer.
  • Prerequisites: Node.js runtime (implied), Homebrew (optional).

Highlighted Details

  • Rule-driven reduction engine with layered rule management (built-in, user config, project config).
  • Supports numerous host integrations for popular coding and agent tools (e.g., Claude Code, Copilot CLI, Cursor, Gemini CLI).
  • Provides diagnostic commands (tokenjuice doctor, tokenjuice verify, tokenjuice stats) for troubleshooting and insights.
  • reduce-json command offers a stable machine protocol for host adapters.

Maintenance & Community

The project is built by Vincent Koc. The README indicates it is a "usable foundation" currently focused on "deeper coverage and tuning," with some integrations noted as "beta."

Licensing & Compatibility

The provided README does not specify a software license. This lack of explicit licensing information may pose compatibility concerns for commercial use or integration into closed-source projects.

Limitations & Caveats

Several integrations are marked as beta, suggesting potential instability or ongoing development. The project's focus on "deeper coverage and tuning" implies that breaking changes or API adjustments may occur. Crucially, the absence of a stated license is a significant adoption blocker for many use cases.

Health Check
Last Commit

3 weeks ago

Responsiveness

Inactive

Pull Requests (30d)
5
Issues (30d)
0
Star History
55 stars in the last 30 days

Explore Similar Projects

Starred by Sebastian Raschka Sebastian Raschka(Author of "Build a Large Language Model (From Scratch)"), Jeff Hammerbacher Jeff Hammerbacher(Cofounder of Cloudera), and
9 more.

qwen-code by QwenLM

0.7%
26k
AI coding agent for complex codebases
Created 1 year ago
Updated 19 hours ago
Feedback? Help us improve.