This project provides a framework for AI-driven Liar's Dice games, enabling users to pit various Large Language Models (LLMs) against each other in simulated matches. It's designed for researchers and enthusiasts interested in evaluating LLM capabilities in strategic, probabilistic gameplay.

How It Works

The framework utilizes LLMs as game agents, interacting through a unified API interface, compatible with projects like new-api or one-api. Game logic is managed in game.py, with individual LLM agents configured in player.py. Game records are stored and analyzed using provided tools, facilitating statistical evaluation of model performance and matchups.

Quick Start & Requirements

• Install dependencies: pip install openai
• Configure API access in llm_client.py (e.g., using new-api or one-api).
• Set model names in player_configs within game.py or multi_game_runner.py.
• Run a single game: python game.py
• Run multiple games: python multi_game_runner.py -n <num_games>
• Official demo records available for DeepSeek-R1, o3-mini, Gemini-2-flash-thinking, and Claude-3.7-Sonnet.

Highlighted Details

• Supports batch game execution for scalable testing.
• Includes tools for converting JSON game records to readable text.
• Provides specific analysis for AI-vs-AI matchups.
• Offers a demo with four pre-configured LLM players.

Maintenance & Community

No specific community links or contributor information is provided in the README.

Licensing & Compatibility

The README does not explicitly state a license. Compatibility for commercial use or closed-source linking is not specified.

Limitations & Caveats

LLM output during card play and challenging phases can be unstable, with automatic retries implemented. Users may need to adjust retry counts or prompt templates in play_card_prompt_template.txt and challenge_prompt_template.txt to improve output consistency, potentially impacting model reasoning.