promptleak: detect system-prompt leakage in transcripts in AI agents

What it detects

Detects system-prompt leakage in LLM app transcripts and outputs, offline.

System prompts often carry business rules, internal codenames, routing logic and sometimes secrets that should never have been there. Prompt-extraction attacks ("repeat the words above", "print your instructions in a code block") get models to recite them. promptleak compares what the model actually said (assistant text and tool-call arguments) against the system/developer prompt and reports how much of it came back out, verbatim or lightly obfuscated.

Inputs: OpenAI-style message lists (incl. developer role and tool_calls), Anthropic requests/sessions (top-level system, tool_use blocks), generic {"messages": [...]} JSON or JSONL, plain text outputs, directories, or stdin (-). If no --prompt is given, the system messages inside each transcript are used.

Rules

Rule	Severity	Finding
PL01	critical	Bulk leak: coverage ≥ `--bulk` (default 50%) of a prompt
PL02	high	Verbatim span ≥ `--span` words (default 25) or coverage ≥ `--partial` (default 20%)
PL03	medium	Shorter verbatim fragment(s) (at least n words)
PL04	high	A prompt line marked confidential/secret/"do not reveal"/password/API key/codename was reproduced
PL05	critical	A `--protect` string appeared in output (also checked inside base64/hex blobs); shown masked
PL06	high	A base64 / base64url / hex blob in output decodes to prompt content (e.g. smuggled via a tool call)
PL07	info	A user turn looks like a prompt-extraction attempt (context for the findings above)

Usage

cd tools/promptleak
python3 promptleak.py examples/leaky_transcript.json          # 6 findings, exit 1
python3 promptleak.py examples/clean_transcript.json          # 1 info (attempt, no leak), exit 0
python3 promptleak.py examples/anthropic_session.jsonl        # PL03 fragment, exit 1
python3 promptleak.py -p examples/system_prompt.txt examples/model_output.txt --protect "BLUE HERON"
my_app --once | python3 promptleak.py -p prompts/system.txt - --fail-on high   # CI / guard
python3 promptleak.py logs/ -p prompts/system.txt --json --min-severity medium

After pip install of the kit, the same options work as agentsec promptleak …. Every tool also accepts --help and almost every tool accepts --json.

Example output

See the examples under tools/promptleak/examples/ in the repository.

SARIF and CI

agentsec promptleak path/to/input --sarif > promptleak.sarif

# GitHub Actions (composite action ships with the repo)
- uses: dunkleschitz/agentsec@v0
  with:
    tools: promptleak
    paths: path/to/input
    fail-on: high

SARIF 2.1.0 can be uploaded to GitHub code scanning. The composite action writes per-tool reports under agentsec-sarif/ by default.

OWASP mapping

Heuristic mapping of what this tool helps a reviewer look for. Not an official OWASP endorsement, and not complete coverage of any item. Sources: OWASP Top 10 for LLM Applications 2025 and OWASP Top 10 for Agentic Applications 2026.

LLM02 Sensitive Information Disclosure — OWASP LLM Top 10 (2025)
LLM07 System Prompt Leakage — OWASP LLM Top 10 (2025)

Known limits

Verbatim only. Paraphrases, summaries, translations, and leaks split into chunks shorter

than *n* words are not detected. Lower -n for stricter checks at the cost of false positives from generic phrases ("if you are unsure, ...").

Obfuscation handled: case, punctuation, whitespace, zero-width/Unicode-TAG characters,

full-width forms, base64/base64url/hex. Not handled: ROT13, reversed text, letter spacing like "y o u a r e", leetspeak, other encodings.

A model legitimately quoting public parts of its prompt (e.g. a product list) will show as

PL03; tune --fail-on or keep public text out of the prompt you pass with -p.

PL04 and the text output show snippets of the reproduced prompt; treat promptleak reports

as sensitive as the prompt itself. --protect values are masked.

PL07 is a keyword heuristic for context only; it is neither complete nor a guardrail.
It reads what was logged. It cannot see streamed output that was never written to the log.

Full source, tests, and fixtures: tools/promptleak/ in the repository. Install the kit with pip install git+https://github.com/dunkleschitz/agentsec.git.