Skip to main content
hc
Submit
toolsFirst-partyReview first Safety · Privacy ✓
Promptfoo logo

Promptfoo

Open-source prompt testing and red-teaming framework for LLM outputs, regressions, evaluations, and security checks.

by Promptfoo·added 2026-04-27·
CLI
HarnessCLI
Review first — review before installing

Open the source and read safety notes before installing.

Citation facts

Source-backed facts for citing this resource, derived directly from the registry — also available as plain text for AI assistants.

Canonical URL
https://heyclau.de/entry/tools/promptfoo
Source URLs
https://www.promptfoo.dev/docs, https://github.com/promptfoo/promptfoo, https://www.promptfoo.dev
Brand
Promptfoo
Brand domain
promptfoo.dev
Brand asset source
brandfetch
Privacy notes
Promptfoo sends your prompts and test inputs to the model providers you configure to run evals and red-team probes; review which providers are used and keep secrets out of test cases.
Author
Promptfoo
Claim status
unclaimed
Last verified
2026-04-27

Decision playbook

Ready to evaluate for your workflow

Signals are comparatively strong, but you should still validate source, privacy posture, and package provenance for your environment.

Required checks are still incomplete. Finish source and safety verification before adopting this resource.

Compare context
Selected

0

Current score

78

Baseline

—

Delta

No baseline selected

No major trust-signal divergence detected in the current selection.

Source and provenance checks

Complete

Confirm ownership and provenance before trusting install instructions.

  • Source link availableRequired

    Open the canonical repository and verify ownership.

    Done

  • Source provenance statusRequired

    Marked as first-party.

    Done

  • Metadata reviewed

    Registry metadata indicates a reviewed listing.

    Done

Safety and privacy checks

Required checks missing

Validate risk disclosures before installation or API wiring.

  • Safety notes presentRequired

    No safety notes listed.

    Pending

  • Privacy notes presentRequired

    Review data handling notes before connecting accounts or secrets.

    Done

  • Trust level risk gateRequired

    Trust level does not block evaluation.

    Done

Package and install checks

Needs review

Check package metadata and artifact integrity signals.

  • Install payload available

    Install or copy payload is available for review.

    Done

  • Package verification flag

    No package verification flag provided.

    Pending

  • Checksum metadata

    No checksum provided for downloaded artifact.

    Pending

Compare-driven decision checks

Needs review

Use compare context to validate trade-offs before adoption.

  • Compare tray has multiple entries

    Add at least one more entry to compare trust differences.

    Pending

  • Baseline comparison available

    No baseline peer selected yet.

    Pending

  • Diverging trust signals identified

    No major trust-signal divergence found.

    Pending

Setup at a glance

Copy & paste

Copy-ready — paste the snippet to get started.

Install command

Not provided

Config snippet

Not provided

Copy snippet

Provided

Prerequisites

None

Platforms

1 listed

Install type

Copy & paste

Adoption plan

Balanced adoption plan

Current risk score 20/100. Use staged verification before broader rollout.

Risk 20
Adoption blockers
  • Safety notes are missing.

Pre-adoption checks

Validate source and review signals before any execution.

  • Confirm source provenanceRequired

    Source URL/provenance metadata is present.

    Done

  • Confirm metadata review state

    Listing has review metadata.

    Done

  • Verify install payload

    Install/config payload exists and can be inspected.

    Done

Security checks

Confirm safety, privacy, and package integrity signals.

  • Review safety notesRequired

    Safety notes missing; review source code paths before execution.

    Pending

  • Review privacy notesRequired

    Privacy notes are present.

    Done

  • Verify package integrity metadata

    No package verification/checksum metadata.

    Pending

Rollout

Adopt in controlled steps based on the selected plan.

  • Run in isolated sandbox firstRequired

    Use a constrained sandbox and observe behavior across multiple tasks.

    Pending

  • Roll out graduallyRequired

    Roll out to a small cohort before wider usage.

    Pending

  • Set monitoring and fallback

    Define rollback path and monitor errors after adoption.

    Pending

Evidence readiness

Evidence readiness matrix · balanced

Missing required evidence: Safety notes. Risk score 21.

Risk 21

Source provenance

Present

Source repository/provenance is listed.

Required in this preset

Metadata review

Present

Review metadata is present.

Required in this preset

Safety notes

Missing

Safety notes are missing.

Required in this preset

Privacy notes

Present

Privacy notes are present.

Optional in this preset

Package integrity

Missing

Package integrity metadata is missing.

Optional in this preset

Install payload

Present

Install payload is available.

Required in this preset

Required gaps: Safety notes

Decision timeline

Decision timeline · balanced

Blocking gaps: Review safety notes. Risk 18.

Risk 18

triage

Confirm source provenanceRequired

Source/provenance metadata is available.

Done

triage

Check metadata review statusRequired

Review metadata is available.

Done

verify

Review safety notesRequired

Safety notes are missing.

Pending

verify

Review privacy notes

Privacy notes are available.

Done

verify

Validate package integrity metadata

Package integrity metadata is missing.

Pending

rollout

Verify install payload and commandsRequired

Install payload is available.

Done

Blockers: Review safety notes

Safety & privacy surface

Safety & privacy surface

1 privacy note across 1 risk area. Review closely: credentials & tokens.

1 area
  • PrivacyCredentials & tokensPromptfoo sends your prompts and test inputs to the model providers you configure to run evals and red-team probes; review which providers are used and keep secrets out of test cases.

Disclosure: heyclaude_pick

Privacy notes

  • Promptfoo sends your prompts and test inputs to the model providers you configure to run evals and red-team probes; review which providers are used and keep secrets out of test cases.

Schema details

Install type
copy
Troubleshooting
No
Source repository stats
Scope
Source repo
Skill and platform metadata
Retrieval sources
https://www.promptfoo.dev/docshttps://docs.garak.ai/https://docs.confident-ai.com/
Tool listing metadata
Pricing
open-source
Disclosure
heyclaude_pick
Application category
DeveloperApplication
Operating system
macOS, Windows, Linux, Web
Full copyable content
## How Promptfoo compares

Promptfoo spans LLM testing and red-teaming; related tools in this directory differ by focus:

| Tool | Focus | Open source | Notable for |
| --- | --- | --- | --- |
| **Promptfoo** | LLM evals + red-teaming | Yes | YAML test cases, assertions, CI-friendly red-team plugins |
| **Garak** | LLM vulnerability scanning | Yes | Broad library of attack probes |
| **DeepEval** | Pytest-style LLM evals | Yes | Unit-test ergonomics and metric library |

Choose Promptfoo for combined CI evals and red-teaming; Garak for dedicated probe-based scanning, or DeepEval for pytest-native evaluation.

## Editorial notes

Promptfoo is a practical fit for teams that want repeatable prompt tests, security checks, and evaluation gates in CI.

## Disclosure

Editorial listing. No paid placement or affiliate link is used.

About this resource

How Promptfoo compares

Promptfoo spans LLM testing and red-teaming; related tools in this directory differ by focus:

Tool Focus Open source Notable for
Promptfoo LLM evals + red-teaming Yes YAML test cases, assertions, CI-friendly red-team plugins
Garak LLM vulnerability scanning Yes Broad library of attack probes
DeepEval Pytest-style LLM evals Yes Unit-test ergonomics and metric library

Choose Promptfoo for combined CI evals and red-teaming; Garak for dedicated probe-based scanning, or DeepEval for pytest-native evaluation.

Editorial notes

Promptfoo is a practical fit for teams that want repeatable prompt tests, security checks, and evaluation gates in CI.

Disclosure

Editorial listing. No paid placement or affiliate link is used.

#prompt-testing#security

Source citations

Add this badge to your README

Show that Promptfoo is listed on HeyClaude. Paste this Markdown into your README — it renders the badge and links back to this page.

Listed on HeyClaude
[![Listed on HeyClaude](https://heyclau.de/badge/tools/promptfoo.svg)](https://heyclau.de/entry/tools/promptfoo)

How it compares

Promptfoo side by side with 2 alternatives on trust, install, platform support, and disclosed safety notes — all from reviewed registry metadata.

Field
Promptfoo logoPromptfoo

Open-source prompt testing and red-teaming framework for LLM outputs, regressions, evaluations, and security checks.

Open dossier
Lakera Guard logoLakera Guard

AI security platform for detecting prompt injection, unsafe content, data leakage, and LLM application abuse.

Open dossier
Garak

Open-source LLM vulnerability scanner for probing model behavior, prompt attack surfaces, and safety failures.

Open dossier
Next steps
Open dossierAPI JSONOpen LLMOpen sourceNewsletterClaim listing
Open dossierAPI JSONOpen LLMOpen sourceNewsletterClaim listing
Open dossierAPI JSONOpen LLMOpen sourceNewsletterClaim listing
Trust
Review statusReviewedMaintainer reviewedReviewedMaintainer reviewedReviewedMaintainer reviewed
Package trustPackage not verifiedPackage not verifiedPackage not verified
Source provenanceSource-backedSource-backedSource-backed
Submitter———
Install riskReview firstReview firstReview first
Notes Safety · Privacy ✓ Safety · Privacy ✓ Safety · Privacy ·
BrandPromptfoo logoPromptfooLakera Guard logoLakera Guard—
Categorytoolstoolstools
Sourcefirst-partysource-backedsource-backed
AuthorPromptfooLakeraNVIDIA
Added2026-04-272026-04-272026-04-27
Platforms
CLI
CLI
CLI
Source repo———
Safety notes— missing— missing— missing
Privacy notes✓Promptfoo sends your prompts and test inputs to the model providers you configure to run evals and red-team probes; review which providers are used and keep secrets out of test cases.✓Lakera Guard inspects prompts and model outputs (sent to its API or self-hosted deployment) to detect injection, unsafe content, and data leakage; review what application traffic is sent for scanning and its data handling before routing production traffic.— missing
Prerequisites— none listed— none listed— none listed
Install———
Config———
Citations
ClaimUnclaimedUnclaimedUnclaimed
Open 3 picks in the interactive comparison tool

Related guides

Source-backed guides for putting this to work.

guides

Auditing MCP Client Configuration Before Team Rollout

Audit MCP client configuration before sharing it with a team.

Review firstSource-backedReview firstAdded 1mo ago
Safety ✓ Privacy ✓by YB0y
guides

Auto Mode Hard-Deny Policies For Safe Automation

Set autoMode.hard_deny rules to block risky actions in auto mode.

Review firstSource-backedReview firstAdded 1mo ago
Safety ✓ Privacy ✓by kiannidev
guides

Claude Code in Regulated Finance Environments

Deploy Claude Code in security-sensitive finance environments.

Review firstSource-backedReview firstAdded 8mo ago
Safety ✓ Privacy ✓by JSONbored

Signals

Loading live community signals…

More like this, weekly

A short, calm digest of reviewed Claude resources. Unsubscribe any time.