4 compared
LLM security & red-teaming tools compared
Tools for probing and protecting LLM applications — vulnerability scanning, runtime guardrails, and adversarial red-teaming — compared.
Open in the interactive comparison tool| Field | Garak Open-source LLM vulnerability scanner for probing model behavior, prompt attack surfaces, and safety failures. Open dossier | Lakera Guard AI security platform for detecting prompt injection, unsafe content, data leakage, and LLM application abuse. Open dossier | Microsoft PyRIT Open-source Python framework from Microsoft for identifying generative AI safety and security risks through automated and human-led red-team assessments. Open dossier | Promptfoo Open-source prompt testing and red-teaming framework for LLM outputs, regressions, evaluations, and security checks. Open dossier |
|---|---|---|---|---|
| Trust | ||||
| Install risk | Review first | Review first | Review first | Review first |
| Notes | Safety · Privacy · | Safety · Privacy ✓ | Safety ✓ Privacy ✓ | Safety · Privacy ✓ |
| Category | tools | tools | tools | tools |
| Source | source-backed | source-backed | source-backed | source-backed |
| Author | NVIDIA | Lakera | Microsoft | Promptfoo |
| Added | 2026-04-27 | 2026-04-27 | 2026-06-03 | 2026-04-27 |
| Platforms | CLI | CLI | CLI | CLI |
| Source repo | — | — | — | — |
| Safety notes | — missing | — missing | ✓PyRIT is intended for responsible security and safety assessment; do not run red-team workflows against systems, accounts, or providers you are not authorized to test. Automated and multi-turn assessment strategies can generate adversarial prompts and risky model outputs, so runs should stay inside approved environments with monitoring and review. Treat scenario datasets, custom converters, scorers, and target connectors as test code that can affect cost, rate limits, model behavior, and downstream reporting. | — missing |
| Privacy notes | — missing | ✓Lakera Guard inspects prompts and model outputs (sent to its API or self-hosted deployment) to detect injection, unsafe content, and data leakage; review what application traffic is sent for scanning and its data handling before routing production traffic. | ✓PyRIT can store prompts, model responses, scores, attack results, conversation history, target metadata, and assessment notes in memory backends such as SQLite or Azure SQL. Provider credentials and endpoint secrets are configured through local PyRIT files and environment-style secret storage, and should not be committed or copied into shared reports. Assessment outputs may contain sensitive system behavior, policy weaknesses, generated harmful text, customer data from test targets, or proprietary prompts. | ✓Promptfoo sends your prompts and test inputs to the model providers you configure to run evals and red-team probes; review which providers are used and keep secrets out of test cases. |
| Prerequisites | — none listed | — none listed |
| — none listed |
| Install | — | — | — | — |
| Config | — | — | — | — |
| Citations | ||||
| Claim | Unclaimed | Unclaimed | Unclaimed | Unclaimed |
More comparisons, weekly
A short, calm digest of reviewed Claude resources. Unsubscribe any time.