Augustus: LLM Vulnerability Scanner

Augustus is a Go-based framework designed for security professionals to systematically test Large Language Models (LLMs) against a wide array of adversarial attacks. It addresses the growing need for robust LLM security by providing over 210 probes covering prompt injection, jailbreaks, data extraction, and other vulnerabilities, enabling users to identify and report on LLM weaknesses in production environments.

How It Works

Augustus employs a pipeline architecture for its scanning process: probe selection, optional prompt transformation (buffing), LLM generation via provider integrations, and detector analysis. Its core advantage lies in its Go-native implementation, offering a single, portable binary with efficient concurrency managed by goroutine pools. This design prioritizes production readiness with built-in rate limiting, retry logic, and timeout handling, differentiating it from research-focused tools. Advanced features include iterative attack engines (PAIR, TAP) and multi-turn conversational engines (Crescendo, GOAT, Hydra, Mischievous User) that maintain conversation history and employ sophisticated strategies for deeper vulnerability discovery.

Quick Start & Requirements

• Installation: Requires Go 1.25.3 or later. Install via go install github.com/praetorian-inc/augustus/cmd/augustus@latest or build from source using make build.
• Prerequisites: Go toolchain (version 1.25.3+). API keys for cloud LLM providers are necessary for testing those services.
• Links: CONTRIBUTING.md for contribution guidelines.

Highlighted Details

• Extensive Coverage: 210+ vulnerability probes across 47 attack categories, and 28 LLM provider integrations (including major cloud providers and local options like Ollama).
• Advanced Detection: Utilizes 90+ detectors, including pattern matching, LLM-as-a-judge, and specialized APIs like HarmJudge and Perspective API.
• Prompt Transformations: Supports 7 types of buff transformations (e.g., encoding, paraphrasing, poeticization) to test evasion techniques.
• Production-Ready: Features concurrent scanning, rate limiting, retry logic, and timeout handling for robust testing.
• Flexible Output: Generates reports in Table, JSON, JSONL, and HTML formats.
• Single Binary: Distributes as a single, dependency-free executable.

Maintenance & Community

The project welcomes contributions, with detailed guidelines provided in CONTRIBUTING.md for adding new probes, detectors, or provider integrations.

Licensing & Compatibility

Augustus is licensed under the Apache 2.0 license. No specific compatibility restrictions for commercial use or closed-source linking are mentioned.

Limitations & Caveats

Requires the Go toolchain for installation and building. Certain probes are designed to generate offensive content for safety filter testing and should only be used in authorized environments.