Promptimal is a command-line tool designed to automatically optimize AI prompts for improved performance. It targets AI engineers, researchers, and content creators who need to refine prompt effectiveness without manual iteration or large datasets. The tool leverages a genetic algorithm to evolve prompts, guided by an LLM-based evaluator, offering a faster path to better prompt engineering.

How It Works

Promptimal employs a genetic algorithm to iteratively refine user-provided prompts. In each iteration, it generates multiple prompt variations (population) based on the current best prompt. These variations are then evaluated by an LLM (or a custom evaluator) to score their effectiveness. The highest-scoring prompts are selected to "breed" the next generation, with mutations and crossovers creating new candidates. This process continues until a satisfactory prompt is found or a set number of iterations is reached, aiming for more efficient and effective prompt discovery.

Quick Start & Requirements

• Install via pipx: pipx install promptimal
• Requires OpenAI API key set as an environment variable: export OPENAI_API_KEY="..."
• Run from terminal: promptimal or promptimal --prompt "..." --improve "..."
• Official documentation and examples are available in the README.

Highlighted Details

• Uses a genetic algorithm for prompt optimization.
• Supports LLM-as-judge evaluation with self-consistency.
• Allows custom evaluation functions via a Python script.
• Tunable hyperparameters include num_iters, num_samples, and threshold.

Maintenance & Community

The project is maintained by shobrook. The roadmap indicates future support for other LLM providers (Anthropic, Groq, Ollama) and enhancements like meta-prompt evolution and synthetic test generation.

Licensing & Compatibility

The repository does not explicitly state a license in the provided README. Users should verify licensing for commercial use or integration into closed-source projects.

Limitations & Caveats

Currently, Promptimal primarily relies on OpenAI's API for evaluation, limiting its use for those without access or preferring other providers. The custom evaluator requires specific script formatting and argument parsing.