Ape is an open-source library designed for AI prompt engineering, offering a modular framework to implement, benchmark, and compare state-of-the-art prompt optimization techniques. It targets researchers and developers seeking to streamline experimentation and foster collaborative advancements in prompt engineering methodologies.

How It Works

Ape provides a unified Trainer class, enabling clean, single-file implementations of various prompt optimization methods. Its modular architecture, particularly within ape-common, facilitates easy extension for custom techniques. The library includes a diverse benchmarking suite covering SQL, reasoning, mathematical, and classification tasks, with plans for more benchmarks.

Quick Start & Requirements

• Install via pip: pip install ape-core
• Requires Python.
• Example usage involves importing Prompt and a Trainer (e.g., FewShotTrainer), defining a Generator and Metric, and calling the train method.
• See Example Experiment Code for detailed tutorials.

Highlighted Details

• Implements techniques from papers like DSPy-MIPRO and EvoPrompt.
• Includes community implementations such as Few-Shot, TextGradient, and Optuna trainers.
• Benchmarks include bird-bench (SQL), gpqa (Reasoning), MATH, boolq (Question Answering), and NYT (Classification).
• Aims to implement every paper on prompt optimization or automated prompt engineering.

Maintenance & Community

• Contributions are welcomed for paper implementations, benchmarks, community research, and general improvements.
• Join the Weavel Community Discord for support and discussion.

Licensing & Compatibility

• Released under the MIT License.
• Permissive license suitable for commercial use and integration into closed-source projects.

Limitations & Caveats

The provided example requires users to implement their own Generator and Metric classes, indicating that core components for training are abstract and need user-defined implementations.