PyRIT
What It Is
PyRIT (Python Risk Identification Toolkit) is Microsoft's open-source framework for AI red teaming. It focuses on multi-turn attack orchestration — running automated conversations with a target to find vulnerabilities.
Repository: github.com/Azure/PyRIT
Key Concepts
| Concept | Description |
|---|---|
| Orchestrator | Manages the attack strategy and conversation flow |
| Target | The AI system being tested |
| Scorer | Evaluates whether an attack succeeded |
| Converter | Transforms prompts (encoding, translation, etc.) |
| Memory | Stores conversation history and results |
Installation
pip install pyrit
Use Cases
- Multi-turn jailbreak automation
- Crescendo attacks (gradual escalation)
- Cross-domain prompt injection testing
- Automated red team campaigns across multiple target configurations
- Scoring and comparing model safety across versions
When to Use PyRIT vs. Garak
| Criteria | Garak | PyRIT |
|---|---|---|
| Single-shot probing | Best | Possible |
| Multi-turn attacks | Limited | Best |
| Custom attack strategies | Moderate | Highly customizable |
| Reporting | Built-in | Custom |
| Learning curve | Low | Medium |