This repository provides a dataset of Python programming puzzles designed to teach and evaluate AI's programming proficiency. It offers a diverse range of problems, from trivial to open research questions, with code-based specifications for unambiguous evaluation. The dataset is valuable for AI research in code generation and self-improvement.

How It Works

The core of the dataset consists of Python functions, each defining a puzzle. A puzzle is solved by providing an input that makes the function return True. This code-based specification eliminates the ambiguity of natural language descriptions and the need for separate test cases. The repository also includes example solutions generated by AI models like OpenAI's Codex, demonstrating the dataset's utility for benchmarking AI performance.

Quick Start & Requirements

• Explore puzzles and AI solutions via the provided Binder link: https://mybinder.org/v2/gh/microsoft/PythonProgrammingPuzzles/main?labpath=intro.ipynb
• Requires Python 3.x.

Highlighted Details

• Dataset spans trivial programming exercises to open problems in computer science and mathematics.
• Includes classic puzzles like Towers of Hanoi, verbal arithmetic, and Game of Life variants.
• Features competitive programming problems from Codeforces and Olympiad problems (ICPC, IMO).
• Contains open problems in graph theory (Conway's 99-graph problem) and number theory (factoring, discrete log).

Maintenance & Community

• Actively maintained by Microsoft researchers.
• Welcomes contributions via pull requests; a CLA is required.
• Follows the Microsoft Open Source Code of Conduct.

Licensing & Compatibility

• The repository itself does not explicitly state a license in the README.
• Compatibility for commercial use or closed-source linking is not specified.

Limitations & Caveats

• The dataset includes open problems, meaning some puzzles may not have known solutions.
• The README mentions that some AI solutions were found after many attempts, indicating potential difficulty for AI models.
• No explicit license is stated, which may pose a barrier for commercial adoption.