Summary

API Relay Audit is a local security auditing tool designed for AI API relays and LLM proxies. It detects critical vulnerabilities such as prompt injection, model substitution, tool-call rewriting, and Web3 wallet risks. The tool targets engineers and users who rely on third-party AI services and need a repeatable, local report to verify relay integrity before production deployment or handling sensitive data. Its primary benefit is providing auditable security assurance without exposing API keys to external web services.

How It Works

The project offers a standalone audit.py script that requires only Python's standard library and curl, ensuring ease of use and inspection. It executes a series of security probes against a user-specified relay URL, analyzing responses for signs of tampering. The core approach involves simulating various attack vectors to identify prompt injection, context manipulation, model identity spoofing, and data leakage. This method prioritizes user privacy and auditability by keeping all operations local and generating detailed, structured Markdown reports.

Quick Start & Requirements

• Primary install/run command:

  curl -sO https://raw.githubusercontent.com/toby-bridges/api-relay-audit/master/audit.py
  python audit.py --key --url --output report.md
  

• Non-default prerequisites: Python standard library, curl. The standalone script has zero external Python package dependencies.
• Links: GitHub Pages: https://toby-bridges.github.io/api-relay-audit, Chinese page: https://toby-bridges.github.io/api-relay-audit/zh/

Highlighted Details

• Detects prompt injection, prompt extraction, instruction override, jailbreak resistance, context truncation, tool-call substitution, error leakage, and SSE stream anomalies.
• Includes specific probes for Web3 wallet safety, such as transfer guidance checks, signed-transaction refusal, and private-key leak refusal, via --profile web3 or --profile full.
• Generates a structured Markdown report detailing per-step findings and a final LOW/MEDIUM/HIGH verdict.
• Offers two distribution modes: a zero-dependency audit.py script and a modular api_relay_audit/ development version.

Maintenance & Community

Key links include GitHub Pages (https://toby-bridges.github.io/api-relay-audit), a Chinese landing page (https://toby-bridges.github.io/api-relay-audit/zh/), ROADMAP.md, CONTRIBUTING.md, SECURITY.md, and the X handle @li9292.

Licensing & Compatibility

• License: AGPL-3.0-only.
• Compatibility: The AGPL-3.0 license mandates that modified network-service deployments must adhere to the same public source-availability standards. This may impose restrictions on commercial use or integration into closed-source systems if deployed as a network service.

Limitations & Caveats

The tool does not provide official certification of relay safety and is not a substitute for manual security reviews or ongoing operational monitoring. Results marked as "inconclusive" (e.g., blocked probes, ambiguous responses) are not considered clean and are explicitly highlighted in the report.