agent-opfor  by KeyValueSoftwareSystems

Adversary emulation for AI agents and LLM applications

Created 3 months ago
551 stars

Top 57.2% on SourcePulse

GitHubView on GitHub
Project Summary

Addresses the critical need for robust testing of AI agents by providing an open-source adversary emulation framework. It empowers engineers, security analysts, and even non-technical team members to proactively identify vulnerabilities in AI agents, LLM applications, and MCP servers through realistic attack simulations, ultimately leading to more secure and reliable AI systems.

How It Works

OPFOR red-teams the entire AI agent surface, including prompts, tool integrations, MCP servers, memory, and multi-turn reasoning. It generates targeted attacks based on industry standards like OWASP Top 10 suites, executes these attacks against the target agent, and employs an LLM judge to evaluate each response for pass/fail criteria and provide reasoning. All attack prompts, requests, responses, and judge verdicts are logged for reproducibility and auditability.

Quick Start & Requirements

  • Primary install: npm install -g @keyvaluesystems/agent-opfor-cli
  • Prerequisites: Requires API keys for supported LLM providers (e.g., OPENAI_API_KEY, GEMINI_API_KEY, ANTHROPIC_API_KEY).
  • Run: Use opfor run for a one-shot scan or opfor setup to create a reusable configuration file for subsequent runs.
  • Links: Website, Docs, GitHub, Browser Extension, Discord.

Highlighted Details

  • Comprehensive coverage of OWASP LLM Top 10, Agentic AI Top 10, MCP Top 10, API Security Top 10, and EU AI Act bias suites.
  • Multiple operational modes including CLI, a no-code browser extension for non-developers, an MCP server mode for AI coding agents, IDE skills, and an SDK for programmatic integration.
  • Trace-aware testing integrates with observability tools like Langfuse and Netra, allowing the LLM judge to analyze internal agent states, tool calls, and reasoning steps, not just the final output.
  • Autonomous "hunt" mode enables adaptive, multi-agent attack campaigns against a given objective and endpoint.

Maintenance & Community

Built by the team at KeyValue Software Systems. Community support is available via Discord. For inquiries, contact contact@agentopfor.ai.

Licensing & Compatibility

Licensed under the Apache 2.0 license. This license permits commercial use and integration into closed-source projects, with standard attribution and license inclusion requirements.

Limitations & Caveats

The autonomous "hunt" mode currently relies exclusively on Claude models for its internal agents, though the target can be any system. Users are advised to use OPFOR only on systems they own or are authorized to test.

Health Check
Last Commit

3 days ago

Responsiveness

Inactive

Pull Requests (30d)
81
Issues (30d)
42
Star History
547 stars in the last 30 days

Explore Similar Projects

Feedback? Help us improve.