tokenjuice  by vincentkoc

Token compaction CLI for agent workflows

Created 1 month ago
343 stars

Top 80.6% on SourcePulse

GitHubView on GitHub
1 Expert Loves This Project
Project Summary

Tokenjuice addresses the problem of excessive, noisy terminal output generated by agent workflows, which wastes context and necessitates reruns. It offers a deterministic output compaction solution for developers and users of terminal-heavy agent and coding frameworks, aiming to reduce transcript waste and improve tool handoffs.

How It Works

Tokenjuice functions as a deterministic output compactor. It observes the output of terminal commands executed by agents, applying rule-driven reducers to generate a smaller, semantically equivalent payload. This approach preserves command semantics, avoids making the shell "magical," and provides inspectable JSON rules for clarity. Host integrations act as thin wrappers, leveraging the core reducer logic rather than implementing custom adapter logic.

Quick Start & Requirements

  • Installation: Install globally via package managers (npm install -g tokenjuice, pnpm add -g tokenjuice, yarn global add tokenjuice) or using Homebrew (brew install tokenjuice).
  • Integrations: Install specific host integrations using tokenjuice install [client_name]. Note: OpenClaw integration requires OpenClaw 2026.4.22 or newer.
  • Prerequisites: Node.js runtime (implied), Homebrew (optional).

Highlighted Details

  • Rule-driven reduction engine with layered rule management (built-in, user config, project config).
  • Supports numerous host integrations for popular coding and agent tools (e.g., Claude Code, Copilot CLI, Cursor, Gemini CLI).
  • Provides diagnostic commands (tokenjuice doctor, tokenjuice verify, tokenjuice stats) for troubleshooting and insights.
  • reduce-json command offers a stable machine protocol for host adapters.

Maintenance & Community

The project is built by Vincent Koc. The README indicates it is a "usable foundation" currently focused on "deeper coverage and tuning," with some integrations noted as "beta."

Licensing & Compatibility

The provided README does not specify a software license. This lack of explicit licensing information may pose compatibility concerns for commercial use or integration into closed-source projects.

Limitations & Caveats

Several integrations are marked as beta, suggesting potential instability or ongoing development. The project's focus on "deeper coverage and tuning" implies that breaking changes or API adjustments may occur. Crucially, the absence of a stated license is a significant adoption blocker for many use cases.

Health Check
Last Commit

1 day ago

Responsiveness

Inactive

Pull Requests (30d)
113
Issues (30d)
1
Star History
196 stars in the last 30 days

Explore Similar Projects

Starred by Jeff Hammerbacher Jeff Hammerbacher(Cofounder of Cloudera), Eric Zhu Eric Zhu(Coauthor of AutoGen; Research Scientist at Microsoft Research), and
8 more.

qwen-code by QwenLM

0.8%
25k
AI coding agent for complex codebases
Created 11 months ago
Updated 11 hours ago
Feedback? Help us improve.