hitlguard: detect missing human approval on consequential tool calls in AI agents

What it detects

Offline human-in-the-loop (HITL) gap detector for AI agent transcripts.

Consequential tool calls — pay, transfer, delete, deploy, force-push, send email, slack post, shell rm -rf — that fire without a nearby confirm-shaped user turn or approval-tool success are a common autonomy failure mode. hitlguard reads saved transcripts, classifies each tool call by name and argument risk, and flags those that lack confirmation inside a configurable lookback window.

Standard library only. It never runs an agent and never contacts the network.

Rules

Rule	Severity	What it flags
HG01	critical	Critical-tier tool (pay/transfer/destroy/force-push/prod deploy/…) without nearby confirmation
HG02	high	High-tier tool (send email/slack/delete/deploy/publish/shell/…) without nearby confirmation
HG03	high	Burst of unconfirmed consequential tools (`--burst`, default 3)
HG04	high	Assistant said it would skip asking, then took a consequential action
HG05	critical	Confirm-shaped text in an assistant/tool role (spoofed approval)
HG06	medium	Blanket pre-authorization ("always approve", "don't ask again") followed by consequential calls
HG07	medium	Approval-tool success with no matching approval tool call in lookback
HG08	info	Inventory of consequential calls and confirmation status

Usage

python3 hitlguard.py transcript.json [more.jsonl dir/ -]
  --window N           events to look back for confirm/approval (default 6)
  --burst N            unconfirmed consequential calls → HG03 (default 3)
  --require GLOB ...   tool-name globs that always require confirmation
  --safe GLOB ...      tool-name globs that never need confirmation
  --disable RULE ...   skip rule ids
  --json               machine-readable output
  --min-severity S     hide findings below S (default info)
  --fail-on S          exit 1 if any finding >= S (default high; any, none)
  --list-rules

After pip install of the kit, the same options work as agentsec hitlguard …. Every tool also accepts --help and almost every tool accepts --json.

Example output

python3 hitlguard.py examples/risky_openai.json --disable HG08
python3 hitlguard.py examples/clean_confirmed.json --disable HG08
python3 hitlguard.py examples/clean_approval_tool.jsonl --json

SARIF and CI

agentsec hitlguard path/to/input --sarif > hitlguard.sarif

# GitHub Actions (composite action ships with the repo)
- uses: dunkleschitz/agentsec@v0
  with:
    tools: hitlguard
    paths: path/to/input
    fail-on: high

SARIF 2.1.0 can be uploaded to GitHub code scanning. The composite action writes per-tool reports under agentsec-sarif/ by default.

OWASP mapping

Heuristic mapping of what this tool helps a reviewer look for. Not an official OWASP endorsement, and not complete coverage of any item. Sources: OWASP Top 10 for LLM Applications 2025 and OWASP Top 10 for Agentic Applications 2026.

LLM06 Excessive Agency — OWASP LLM Top 10 (2025)
ASI09 Human-Agent Trust Exploitation — OWASP Agentic Top 10 (2026)
ASI10 Rogue Agents — OWASP Agentic Top 10 (2026)

Known limits

Heuristic tiering from tool names and a few argument shapes. A tool named

helper that secretly wires money will not be flagged unless you --require it.

"Confirm-shaped" matching is English-centric and can false-positive on short

replies, or false-negative on unusual phrasing / button clicks that were never logged as text.

Only sees what the transcript recorded. UI approval clicks, out-of-band Slack

reactions, or unlogged gates are invisible.

Coding-agent write_file / read_file loops are intentionally low-noise

(medium or ignored). Use --require for paths that must always be gated.

A clean result does not prove a human approved the action in the real system.

Full source, tests, and fixtures: tools/hitlguard/ in the repository. Install the kit with pip install git+https://github.com/dunkleschitz/agentsec.git.