This repository provides a set of AI Red Teaming challenges and infrastructure, based on Chat Copilot, designed for security professionals to systematically test AI systems. It offers a practical learning environment for identifying novel adversarial machine learning and Responsible AI failures beyond traditional security vulnerabilities.

How It Works

The playground environment leverages a modified Chat Copilot instance for each challenge, presenting users with specific objectives. These challenges simulate real-world scenarios, requiring users to employ techniques like prompt injection, metaprompt extraction, and guardrail bypasses to achieve defined goals, such as credential exfiltration or generating restricted content. The architecture is containerized using Docker Compose for ease of deployment.

Quick Start & Requirements

• Install/Run: docker-compose up
• Prerequisites: Docker, Python 3.8+, Azure OpenAI Endpoint with an API key.
• Setup: Requires setting Azure OpenAI endpoint and API key via .env file.
• Docs: AI Red Teaming Playground Labs

Highlighted Details

• Offers 12 distinct challenges ranging from Level 1 to Level 3, covering various AI security failure modes.
• Includes examples of direct and indirect prompt injection, metaprompt extraction, and guardrail bypasses.
• Challenges can be customized by modifying challenges/challenges.json and regenerating the Docker Compose configuration.
• The core environment is built upon the Chat Copilot project, with modifications for lab purposes.

Maintenance & Community

• Developed by security professionals from Microsoft, including contributors to the PyRIT project.
• Community support available via PyRIT Discord.
• Related tool: PyRIT.

Licensing & Compatibility

• License: MIT License.
• Compatibility: Suitable for commercial use and integration with closed-source projects.

Limitations & Caveats

The repository's challenges are no longer updated for future courses, meaning they may not reflect the latest AI vulnerabilities or defenses. While CTFd and chat-score components are included for reference, they are not utilized in the default Docker Compose setup, limiting built-in progress tracking and scoring for self-hosted instances.