Most “AI security tools” do different jobs. Some probe a live model. Some lint agent source code. Some discover and scan MCP servers on your laptop. Some sit in the request path and filter prompts. agentsec does none of those. It reads files you already have — transcripts, MCP client configs, tool-definition snapshots, prompt templates, memory dumps — and emits findings without a network, a model, or a third-party package.
That is a narrow job, and it is intentionally so. The points below are differences, not rankings. Several of these projects are more mature, more widely used, or more complete in their own category than agentsec is in theirs.
At a glance
| Project | License | What it inspects | Needs a live model or server? | Data leaves the machine? | Third-party deps |
|---|---|---|---|---|---|
| agentsec | MIT | Saved transcripts, MCP client configs, tool lockfiles, templates, memory dumps, some source-adjacent files | No | No. The tools do not open a network connection. | None (Python standard library only) |
Snyk Agent Scan (formerly Invariant Labs mcp-scan) |
Apache-2.0 | Installed agent components: MCP servers, skills, and client configs (Claude, Cursor, Copilot, Windsurf, Gemini CLI, and others) | Scan analysis uses Snyk’s API. The scanner may also start configured stdio MCP servers, which the project recommends doing in a sandbox. | Yes, for a scan: component information (tool names and descriptions, server configs and signatures, skill content; secrets redacted) is sent to the Agent Scan API. A Snyk account and SNYK_TOKEN are required. The project states it does not store or log MCP tool-call contents or results. |
Distributed as a Python package / binary; not stdlib-only |
| Cisco MCP Scanner | Apache-2.0 | MCP servers and tools (descriptions, prompts, resources, and optionally server source and dependencies) | Can connect to a live MCP server, or run in a documented static/offline mode on pre-generated JSON. Optional engines call Cisco AI Defense, an LLM, or VirusTotal. | Depends on the engines you enable. YARA / static / readiness checks can stay local; cloud and VirusTotal analyzers send data to those APIs. | Yes (YARA and optional API clients, pip-audit, and related extras) |
| OWASP Agentic Threat Auditor (OATA) | Apache-2.0 | A running agent’s tool-dispatch loop (LangChain, LangGraph, CrewAI, AutoGen, MCP adapters) | Yes — it is a dynamic red-team toolkit, not a file scanner. It defaults to localhost / local mocks. | Intended to stay on the local mock environment you point it at; it is not a static offline linter. | Yes (framework adapters and test runtime) |
| garak (NVIDIA) | Apache-2.0 | A live LLM or dialog system, via dozens of probes (jailbreaks, injection, leakage, toxicity, and more) | Yes. It sends prompts to the target model and scores the generations. | Prompt traffic goes to the model provider you configure (or a local model). It is not a transcript linter. | Yes |
| promptfoo | MIT | LLM apps, agents, and RAG systems, via evals and red-team plugins | Yes, for evals and red teams — it calls the providers you configure. There is also a code-scanning feature for LLM-related issues in a repo. | The project documents that core evals run locally and talk to your configured provider. Hosted red-team generation and other cloud-backed features are opt-in (or default for some red-team paths); review their current data-handling docs. | Yes (Node.js CLI; also usable via npx / brew / a Python wrapper) |
| LLM Guard (Protect AI / Palo Alto Networks) | MIT | Live prompts and outputs, in-process (injection, PII, secrets, toxicity, and other input/output scanners) | It is a runtime library, not an offline file audit. Several scanners load ML models. | Runs in your process. Model weights are additional artifacts; it is not a zero-dependency CLI. | Yes (and local ML models) |
| Agent Audit | MIT | Agent source code and MCP configs: prompt-injection sinks, taint into exec/shell/SQL, secrets, tool-schema and HITL issues, with rules mapped to the OWASP Agentic Top 10 (2026) |
No (static analysis) | No, as a local static scanner | Yes (packaged as agent-audit on PyPI) |
OWASP Agentic AI Scanner (owasp-scan) |
MIT | A source tree, for risks from the OWASP Top 10 for Agentic AI Applications (AA01–AA10) | No (static analysis). Emits SARIF. | No, as a local static scanner | Yes (PyPI package) |
| LLMSecTest | MIT | A running LLM app (black-box) and, optionally, a repo, model file, or vector store (white-box), as pytest tests mapped to the OWASP LLM Top 10 | Black-box tests need a target app. Some white-box checks (for example a pickle opcode scan) are offline. | Traffic goes to the target you name. The project is designed to run in your own pytest suite. | Yes (pytest-based package) |
When agentsec is a good fit
- You already save agent transcripts, MCP client configs, or
tools/listsnapshots, and you want a CI gate on those files. - The scan has to run in an air-gapped job, a distroless image, or any environment where
pip installof a stack of AI libraries is unwelcome. The tools are single Python files and the kit’s only runtime requirement is Python 3.10+. - You need a specific check that source linters and model probes do not cover well: secret flow from a tool result into a later webhook, tool-result prompt injection, MCP tool-definition drift against a lockfile, least-privilege grades of a Cursor or Claude Desktop config, or PII in a shared log.
- You want SARIF or JSON from a composite GitHub Action, or a pre-commit hook, without standing up a model or an account.
When you should use something else
- You need to know whether the model can be jailbroken. That is garak, promptfoo, or LLMSecTest. agentsec never talks to a model. A clean transcript scan does not mean the agent would resist a new payload tomorrow.
- You need to break a live tool-calling loop. OATA is built for dynamic testing of excessive agency and parameter-mutation bypasses. agentsec only sees what was logged.
- You want to discover every MCP server and skill already installed on a workstation. Snyk Agent Scan auto-discovers those layouts. agentsec will grade a config or a snapshot you hand it, and it will not start the server to see what it really exposes. If sending redacted component metadata to Snyk is acceptable in your environment, Agent Scan covers a discovery problem agentsec does not.
- You want a deep, live, or multi-engine look at an MCP server itself (YARA rules, optional LLM review, dependency CVEs, binaries). Cisco MCP Scanner is aimed at that, including an offline mode if you cannot leave the network.
- You want to lint the Python (or similar) that implements the agent — taint into
exec, unpinned tools in code, missing HITL in source. Look at Agent Audit or owasp-agentic-scanner. agentsec’s closest source-adjacent tools (vibeaudit,mcpsink,tmplfence,schemafence) are narrower heuristic scanners, not a full taint engine. - You need an in-process filter in front of the model. That is LLM Guard (or a commercial gateway). agentsec is an audit tool you run on artifacts, not a runtime guardrail.
Honest limits of agentsec
- It is experimental and AI-built. The tests are extensive against synthetic, fake fixtures. The kit has not been benchmarked against public attack corpora or every vendor’s live schema.
- Detectors are regex, AST, and value-equality heuristics. They produce false positives and false negatives. A clean result is not a proof of safety.
- Transcript tools only see what was logged. Unlogged tool calls, transformed secrets, and servers that show a different
tools/listto another client are out of scope. - It does not replace code review, dependency scanning, secret scanning of the repo itself, or a red-team engagement.
Using them together
A reasonable split for a team that already ships agents:
- Lint agent source and MCP server code in CI (Agent Audit or owasp-agentic-scanner).
- Grade checked-in client configs and pin
tools/listwith agentsec (mcpposture,toolpin,toolpoison). - On every saved run, scan the transcript with agentsec (
resultguard,secretflow,piiguard,pathfence,shellfence,fetchfence). - Periodically red-team the live agent with garak, promptfoo, or OATA.
- If you need a request-path filter, add a runtime guard such as LLM Guard.
Those layers overlap on purpose. agentsec is the offline, zero-dependency layer in the middle of that list — not the whole list.
Sources
Project behavior, licenses, and data-handling notes above are taken from the public READMEs and docs linked here, read on 9 November 2026:
- github.com/snyk/agent-scan and the snyk-agent-scan package description (token requirement, API upload, redaction, stdio-server caveat)
- github.com/cisco-ai-defense/mcp-scanner
- github.com/OWASP/Agentic-Threat-Auditor
- github.com/NVIDIA/garak
- github.com/promptfoo/promptfoo and promptfoo FAQ
- github.com/protectai/llm-guard
- github.com/HeadyZhang/agent-audit
- github.com/NP-compete/owasp-agentic-scanner
- llmsec.dev
If a project’s current docs disagree with this page, trust those docs and please open an issue. agentsec is not affiliated with the other projects named here.