Promptfoo is an open-source security testing framework designed specifically for LLM applications, agents, and RAG pipelines. It enables automated red teaming by simulating adversarial attacks to uncover vulnerabilities such as prompt injections, jailbreaks, and PII leaks.
Key Features
- Automated red teaming simulations for agents and RAG architectures.
- Detection of direct and indirect prompt injections and custom jailbreaks.
- Security testing for insecure tool use and business rule violations in AI agents.
- Integration with CI/CD pipelines (GitHub, GitLab, Jenkins) for continuous security.
- Support for 50+ LLM vulnerability types including toxicity and data leakage.
Use Cases
- Automated Red Teaming: Generating thousands of context-aware attacks tailored to specific AI application logic.
- CI/CD Security Scanning: Finding LLM vulnerabilities in pull requests before deployment.
- Evaluation & Benchmarking: Testing prompts, models, and RAG pipelines for factuality and security.
