arc_pi_taxonomy  by Arcanum-Sec

A structured taxonomy for prompt injection attacks

Created 10 months ago
410 stars

Top 71.2% on SourcePulse

GitHubView on GitHub
Project Summary

<Arcanum-Sec/arc_pi_taxonomy provides a structured taxonomy of prompt injection attacks, categorizing attack intents, techniques, and evasions. It serves as a vital resource for security researchers, AI developers, and red teamers seeking to understand and mitigate risks in AI-driven applications. The taxonomy offers a clear, organized framework for analyzing adversarial AI behaviors and enhancing system security.>

How It Works

The taxonomy is organized into key areas: Attack Intents, Attack Techniques, and Attack Evasions, with detailed descriptions available in individual Markdown files within respective folders. This structured approach facilitates a systematic analysis of prompt injection attack vectors, enabling a deeper comprehension of their objectives, methodologies, and obfuscation tactics. This organization is advantageous for researchers and developers aiming to build robust defenses against evolving AI security threats.

Quick Start & Requirements

This repository functions as a knowledge base. Users can access detailed information by navigating the Markdown files within the attack_intents, attack_techniques, and attack_evasions folders. No specific installation or execution commands are provided, as it is a documentation resource.

Highlighted Details

  • Includes a structured assessment framework for evaluating the security of AI-enabled infrastructures.
  • Features a curated list of example probes for identifying AI-enabled web forms and endpoints.
  • Categorizes attack intents, techniques, and evasions for comprehensive analysis.

Maintenance & Community

The repository is in active development, with contributions, corrections, and additions actively encouraged. Community engagement and feedback are welcomed to evolve the taxonomy.

Licensing & Compatibility

The content is licensed under a Creative Commons Attribution 4.0 International License (CC BY 4.0). Use, reference, or building upon the methodology requires explicit attribution to "Arcanum Prompt Injection Taxonomy by Jason Haddix (Arcanum Information Security)". This license generally permits commercial use and derivative works, provided attribution is maintained.

Limitations & Caveats

The repository is explicitly stated to be in active development, indicating that the taxonomy may be incomplete or subject to change. As a classification system, it does not provide direct tools or code for executing or mitigating attacks, requiring users to interpret and apply the information.

Health Check
Last Commit

1 month ago

Responsiveness

Inactive

Pull Requests (30d)
0
Issues (30d)
0
Star History
63 stars in the last 30 days

Explore Similar Projects

Starred by Dan Guido Dan Guido(Cofounder of Trail of Bits), Chip Huyen Chip Huyen(Author of "AI Engineering", "Designing Machine Learning Systems"), and
5 more.

PurpleLlama by meta-llama

0.3%
4k
LLM security toolkit for assessing/improving generative AI models
Created 2 years ago
Updated 2 days ago
Feedback? Help us improve.