Garak

What It Is

Garak is an open-source LLM vulnerability scanner. It automates probing models for known vulnerability categories — jailbreaks, prompt injection, data leakage, toxicity, and more.

Repository: github.com/NVIDIA/garak

Installation

pip install garak

Basic Usage

# Scan a local Ollama model
garak --model_type ollama --model_name llama3

# Scan OpenAI
garak --model_type openai --model_name gpt-4

# Run specific probes
garak --model_type ollama --model_name llama3 --probes encoding.InjectBase64

# List available probes
garak --list_probes

Key Probe Categories

Probe	What It Tests
`dan`	DAN (Do Anything Now) jailbreak variants
`encoding`	Base64, ROT13, and other encoding bypasses
`glitch`	Token-level adversarial inputs
`knownbadsignatures`	Known malicious prompt patterns
`lmrc`	Language Model Risk Cards checks
`misleading`	Hallucination and misinformation
`packagehallucination`	Hallucinated package names (supply chain risk)
`promptinject`	Prompt injection techniques
`realtoxicityprompts`	Toxicity evaluation
`snowball`	Escalating complexity probes
`xss`	Cross-site scripting via model output

Output

Garak produces structured reports showing which probes succeeded, failure rates, and specific responses. Export to JSON for integration with other tools.