This repository provides a Python library for estimating the token count and associated costs of using various Large Language Models (LLMs). It aims to help developers building AI applications and agents accurately track and manage their LLM API expenses.

How It Works

The library leverages tiktoken, OpenAI's official tokenizer, for accurate tokenization of text and message formats. For Anthropic models (Claude 3.5 and newer), it utilizes the Anthropic beta token counting API. For older Claude models, it falls back to approximating token counts using tiktoken with the cl100k_base encoding. The cost calculations are based on a comprehensive, regularly updated list of model prices stored in model_prices.json.

Quick Start & Requirements

• Installation: pip install tokencost
• Prerequisites: Python 3.x. No other specific hardware or software dependencies are mentioned.
• Usage: Import calculate_prompt_cost, calculate_completion_cost, count_message_tokens, or count_string_tokens functions.
• Documentation: README

Highlighted Details

• Supports over 400 LLMs, including OpenAI, Anthropic, Mistral, Cohere, and Google Gemini models.
• Provides functions for both string-based and message-list-based token counting and cost estimation.
• Includes detailed pricing information for prompt and completion tokens across a wide range of models.

Maintenance & Community

• The project is maintained by AgentOps-AI.
• Community engagement is encouraged via Twitter and Discord.

Licensing & Compatibility

• License: MIT License.
• Compatibility: Permissive MIT license allows for commercial use and integration into closed-source projects.

Limitations & Caveats

• While the README states it supports "Major LLM providers," the specific list of supported models is extensive and may evolve. The accuracy for newer or less common models relies on the maintenance of model_prices.json.