arc_pi_taxonomy  by Arcanum-Sec

A structured taxonomy for prompt injection attacks

Created 6 months ago
253 stars

Top 99.4% on SourcePulse

GitHubView on GitHub
Project Summary

<Arcanum-Sec/arc_pi_taxonomy provides a structured taxonomy of prompt injection attacks, categorizing attack intents, techniques, and evasions. It serves as a vital resource for security researchers, AI developers, and red teamers seeking to understand and mitigate risks in AI-driven applications. The taxonomy offers a clear, organized framework for analyzing adversarial AI behaviors and enhancing system security.>

How It Works

The taxonomy is organized into key areas: Attack Intents, Attack Techniques, and Attack Evasions, with detailed descriptions available in individual Markdown files within respective folders. This structured approach facilitates a systematic analysis of prompt injection attack vectors, enabling a deeper comprehension of their objectives, methodologies, and obfuscation tactics. This organization is advantageous for researchers and developers aiming to build robust defenses against evolving AI security threats.

Quick Start & Requirements

This repository functions as a knowledge base. Users can access detailed information by navigating the Markdown files within the attack_intents, attack_techniques, and attack_evasions folders. No specific installation or execution commands are provided, as it is a documentation resource.

Highlighted Details

  • Includes a structured assessment framework for evaluating the security of AI-enabled infrastructures.
  • Features a curated list of example probes for identifying AI-enabled web forms and endpoints.
  • Categorizes attack intents, techniques, and evasions for comprehensive analysis.

Maintenance & Community

The repository is in active development, with contributions, corrections, and additions actively encouraged. Community engagement and feedback are welcomed to evolve the taxonomy.

Licensing & Compatibility

The content is licensed under a Creative Commons Attribution 4.0 International License (CC BY 4.0). Use, reference, or building upon the methodology requires explicit attribution to "Arcanum Prompt Injection Taxonomy by Jason Haddix (Arcanum Information Security)". This license generally permits commercial use and derivative works, provided attribution is maintained.

Limitations & Caveats

The repository is explicitly stated to be in active development, indicating that the taxonomy may be incomplete or subject to change. As a classification system, it does not provide direct tools or code for executing or mitigating attacks, requiring users to interpret and apply the information.

Health Check
Last Commit

1 month ago

Responsiveness

Inactive

Pull Requests (30d)
0
Issues (30d)
0
Star History
59 stars in the last 30 days

Explore Similar Projects

Starred by Chip Huyen Chip Huyen(Author of "AI Engineering", "Designing Machine Learning Systems"), Michele Castata Michele Castata(President of Replit), and
3 more.

rebuff by protectai

0.4%
1k
SDK for LLM prompt injection detection
Created 2 years ago
Updated 1 year ago
Starred by Chip Huyen Chip Huyen(Author of "AI Engineering", "Designing Machine Learning Systems"), Carol Willing Carol Willing(Core Contributor to CPython, Jupyter), and
3 more.

llm-security by greshake

0.1%
2k
Research paper on indirect prompt injection attacks targeting app-integrated LLMs
Created 2 years ago
Updated 2 months ago
Starred by Dan Guido Dan Guido(Cofounder of Trail of Bits), Chip Huyen Chip Huyen(Author of "AI Engineering", "Designing Machine Learning Systems"), and
5 more.

PurpleLlama by meta-llama

0.6%
4k
LLM security toolkit for assessing/improving generative AI models
Created 1 year ago
Updated 1 day ago
Feedback? Help us improve.