garak, LLM vulnerability scanner
Generative AI Red-teaming & Assessment Kit
garak is a command-line tool designed to check if an LLM can be made to fail in undesirable ways. It functions similarly to nmap or Metasploit, but for LLMs.
Key features:
- Vulnerability Probing: Detects hallucination, data leakage, prompt injection, misinformation, toxicity, and jailbreaks.
- Flexible LLM Support: Compatible with Hugging Face Hub, Replicate, OpenAI API, litellm, gguf models, and REST endpoints.
- Customizable Probes: Allows specifying probe families or individual plugins for targeted testing.
- Detailed Reporting: Generates JSONL reports with probing attempts and evaluation results, including hit logs for identified vulnerabilities.
- Extensible Plugin Architecture: Supports developing custom probes and detectors.
Use cases:
- Security assessments of LLMs and dialog systems.
- Red teaming to identify failure modes.
- Evaluating the effectiveness of different LLM configurations.
- Continuous integration testing for LLM-powered applications.
