prompt-hacker-collections  by yunwei37

LLM prompt hacking and defense resource

Created 2 years ago
283 stars

Top 92.4% on SourcePulse

GitHubView on GitHub
Project Summary

Summary

This repository provides a curated collection of prompt adversarial techniques, focusing on prompt injection attacks, defenses, and reverse engineering examples. It serves as a valuable resource for researchers, students, and security professionals seeking to understand and practice LLM security vulnerabilities and mitigation strategies.

How It Works

The project organizes various prompt types, including jailbreaks, reverse engineering, attacks, and defenses, primarily in YAML format for easy parsing and application. It details concepts and provides concrete examples, such as the "DAN 11.0" jailbreak prompt, illustrating methods to bypass LLM restrictions and explore their security boundaries.

Quick Start & Requirements

This repository primarily serves as a reference collection rather than a runnable application. Installation instructions or specific software/hardware prerequisites are not detailed. Users are expected to leverage the provided prompt examples directly within their LLM interaction environments. Relevant external resources on LLM safety are linked.

Highlighted Details

  • Comprehensive categories: Prompt reverse engineering, jailbreaking, attacks, and defense strategies.
  • Structured data: Prompt examples are organized in YAML format for programmatic use.
  • In-depth examples: Features detailed jailbreak prompts like "DAN 11.0" and reverse engineering techniques for models like Notion AI and Midjourney.
  • External resources: Links to OpenAI's safety best practices and Microsoft's LLM red-teaming guides.

Maintenance & Community

The project encourages community contributions through issues and pull requests, fostering collaborative development. Specific details regarding active maintainers, community channels (like Discord/Slack),

Health Check
Last Commit

10 months ago

Responsiveness

Inactive

Pull Requests (30d)
0
Issues (30d)
0
Star History
15 stars in the last 30 days

Explore Similar Projects

Starred by Chip Huyen Chip Huyen(Author of "AI Engineering", "Designing Machine Learning Systems"), Michele Castata Michele Castata(President of Replit), and
3 more.

rebuff by protectai

0.1%
1k
SDK for LLM prompt injection detection
Created 2 years ago
Updated 1 year ago
Starred by Chip Huyen Chip Huyen(Author of "AI Engineering", "Designing Machine Learning Systems"), Elie Bursztein Elie Bursztein(Cybersecurity Lead at Google DeepMind), and
3 more.

llm-guard by protectai

0.9%
2k
Security toolkit for LLM interactions
Created 2 years ago
Updated 3 weeks ago
Starred by Dan Guido Dan Guido(Cofounder of Trail of Bits), Chip Huyen Chip Huyen(Author of "AI Engineering", "Designing Machine Learning Systems"), and
5 more.

PurpleLlama by meta-llama

0.3%
4k
LLM security toolkit for assessing/improving generative AI models
Created 2 years ago
Updated 2 days ago
Feedback? Help us improve.