How Promptfoo Red-Teams LLM Applications in CI/CD
Summary
The article describes Promptfoo’s open-source LLM red-teaming workflow: YAML-configured tests use 50+ plugins and delivery strategies, run in CI/CD, and report findings with an LLM judge and framework mappings. It says OpenAI acquired the project in March 2026.
Key points
- Promptfoo is described as a CLI and Node.js library for evaluating and red-teaming LLM applications, with YAML configuration and CI/CD integration.
- The article says its 50+ plugins target issues including prompt injection, jailbreaks, PII leakage, SSRF, SQL injection, and excessive agency.
- Delivery strategies include prompt-injection framing, jailbreak techniques, and multi-turn escalation using Crescendo.
- The described workflow runs generated tests against a target and uses an LLM judge to grade responses.
- The web UI presents prompts, model responses, grader explanations, and mappings to OWASP LLM Top 10, NIST AI RMF, and MITRE ATLAS.
- The article reports that OpenAI acquired Promptfoo on March 9, 2026, and says the project remains MIT licensed and open source.
Article Details
- Topic
- Promptfoo's LLM red-teaming and security testing capabilities in developer CI/CD pipelines.
Vendors
Tools
GarakGarak gives us the broad CLI sweep across known probe families.PromptfooPromptfoo is an open-source CLI and Node.js library for evaluating and red teaming LLM applications, with YAML configs, 50+ attack plugins, and a web UI that shows where models fail.PyRITPyRIT runs the surgical multi-turn follow-up.