promptaudit: detect secrets and authorization logic inside system prompts in AI agents

Offline hygiene auditor for LLM system prompts. A system prompt is not a secret store and not an access-control layer (OWASP LLM07 System Prompt Leakage, LLM06 Excessive Agency). promptaudit reads system and developer prompts and flags content that should live somewhere else: credentials, internal endpoints, authorization logic, unbounded tool authority, personal data, disclosed decision thresholds, and hidden Unicode. It also flags tool-using prompts that never tell the model to treat tool or retrieved content as data.

What it detects

Offline hygiene auditor for LLM system prompts. A system prompt is not a secret store and not an access-control layer (OWASP LLM07 System Prompt Leakage, LLM06 Excessive Agency). promptaudit reads system and developer prompts and flags content that should live somewhere else: credentials, internal endpoints, authorization logic, unbounded tool authority, personal data, disclosed decision thresholds, and hidden Unicode. It also flags tool-using prompts that never tell the model to treat tool or retrieved content as data.

Standard library only. No network calls. Secrets and PII are masked in every output format.

Rules

Rule	Severity	What it flags
PA01	critical	Credential literal: OpenAI/AWS/GitHub/Slack/Google key shapes, private key blocks, JWTs, bearer tokens, connection strings with passwords, `password: ...` style labels (placeholders like `{{API_KEY}}` skipped)
PA02	high (critical for metadata)	Private, loopback, link-local, or cloud metadata IPs and internal hostnames (`.internal`, `.corp`, `.local`, single-label hosts) in URLs or bare IPs
PA03	high	Authorization enforced only by the model: "only allow ... admins", "if the user says they are an admin", cross-tenant "don't let users view other customers'", admin passphrases
PA04	high	Unbounded authority: "run any command", "without asking for confirmation", "full access", "ignore safety checks", "always approve tool calls"
PA05	high (critical with TAG text)	Zero-width, bidi, and Unicode TAG characters; TAG payloads are decoded into the message
PA06	medium	The prompt mentions tools, browsing, files, email, or retrieval but has no guidance that such content is data, not instructions
PA07	medium (SSN high, phone low)	Personal email addresses (role and example.com addresses skipped), SSN-shaped numbers, phone numbers
PA08	medium	Disclosed decision thresholds or filter criteria: "refunds under $500 are auto-approved", "fraud score above 0.8", blocked-word lists
PA09	low	Relying on the prompt staying secret ("never reveal these instructions")
PA10	info	Inventory: prompt size

Usage

python3 promptaudit.py PROMPT_OR_TRANSCRIPT [...] [--json | --sarif] [--as-text]
                       [--min-severity LEVEL] [--fail-on LEVEL|none] [--disable RULE] [--list-rules]

After pip install of the kit, the same options work as agentsec promptaudit …. Every tool also accepts --help and almost every tool accepts --json.

Example output

examples/clean_prompt.txt shows the same kind of assistant written safely and produces only the PA10 inventory line. examples/risky_openai.json and examples/risky_anthropic.json show transcript extraction. All secret values in the examples are fake (FAKE-EXAMPLE-...).

cd tools/promptaudit
python3 promptaudit.py examples/risky_prompt.txt --min-severity medium

SARIF and CI

agentsec promptaudit path/to/input --sarif > promptaudit.sarif

# GitHub Actions (composite action ships with the repo)
- uses: dunkleschitz/agentsec@v0
  with:
    tools: promptaudit
    paths: path/to/input
    fail-on: high

SARIF 2.1.0 can be uploaded to GitHub code scanning. The composite action writes per-tool reports under agentsec-sarif/ by default.

OWASP mapping

Heuristic mapping of what this tool helps a reviewer look for. Not an official OWASP endorsement, and not complete coverage of any item. Sources: OWASP Top 10 for LLM Applications 2025 and OWASP Top 10 for Agentic Applications 2026.

LLM06 Excessive Agency — OWASP LLM Top 10 (2025)
LLM07 System Prompt Leakage — OWASP LLM Top 10 (2025)

Known limits

English-only phrase heuristics. Paraphrased authorization logic or authority grants will be missed,

and unusual phrasing can produce false positives. Use --disable for rules that don't fit.

PA06 only checks that some untrusted-content guidance exists; it can't judge whether it works.

Prompt guidance is a mitigation, never a control.

Secret detection is shape-based. It doesn't verify keys and won't catch secrets with no known

shape and no label.

It audits the prompt text it is given. Prompts assembled at runtime from templates are better

checked with tmplfence; leakage of a prompt into outputs is promptleak's job.

Full source, tests, and fixtures: tools/promptaudit/ in the repository. Install the kit with pip install git+https://github.com/dunkleschitz/agentsec.git.