This repository provides the official code for the "Solo Performance Prompting" (SPP) paper, enabling Large Language Models (LLMs) to achieve enhanced task-solving capabilities through multi-persona self-collaboration. It is designed for researchers and practitioners interested in advanced LLM agent architectures and cognitive synergy.

How It Works

SPP implements a novel self-collaboration framework where an LLM adopts multiple personas to iteratively refine its problem-solving approach. This multi-persona strategy aims to mimic human cognitive processes, allowing the model to explore diverse perspectives and generate more robust solutions compared to single-prompt methods. The approach is demonstrated across creative writing, collaborative games, and logic puzzles.

Quick Start & Requirements

• Install dependencies: pip install -r requirements.txt
• Configure OpenAI API keys via config_template.sh (Azure API used in experiments).
• Run tasks using provided bash scripts:
  • Trivia Creative Writing: bash scripts/trivia_creative_writing.sh
  • Codenames Collaborative: bash scripts/codenames_collaborative.sh
  • Logic Grid Puzzle: bash scripts/logic_grid_puzzle.sh
• Prompts, datasets, and experimental results are available in respective folders (prompts/, data/, logs/).

Highlighted Details

• Code and results for GPT-3.5 and Llama2 inference, demonstrating emergent cognitive synergy.
• Accepted as a main conference paper at NAACL2024.
• Detailed log file formats include evaluation metrics, API call details, parsing success flags, and token usage.

Maintenance & Community

• The project is associated with authors Zhenhailong Wang, Shaoguang Mao, Wenshan Wu, Tao Ge, Furu Wei, and Heng Ji.
• The codebase structure references the Tree-of-thought official repo.

Licensing & Compatibility

• The repository does not explicitly state a license. Users should verify licensing for commercial or closed-source use.

Limitations & Caveats

The project primarily focuses on demonstrating the SPP methodology with OpenAI's API and Llama2, and may require adaptation for other LLM providers or models. Specific configurations for hyperparameter tuning are based on GPT-4 experiments.