Building Custom Tooling

When to Build Custom

Build custom when:

• Existing tools don't support your target's specific API or interface
• You need multi-turn strategies that existing orchestrators can't express
• You're testing proprietary tool-use integrations
• You want tighter integration with your existing pentest workflow

Minimal Architecture

Your Local LLM (attacker brain)
        ↕
Orchestration Script (Python)
        ↕
Target AI System (API/Web)
        ↕
Logger (everything gets saved)

Core Components

Target Adapter

Handles communication with the target:

import requests

class TargetAdapter:
    def __init__(self, api_url, api_key):
        self.url = api_url
        self.headers = {"Authorization": f"Bearer {api_key}"}
    
    def send(self, message, conversation_id=None):
        payload = {"message": message}
        if conversation_id:
            payload["conversation_id"] = conversation_id
        response = requests.post(self.url, json=payload, headers=self.headers)
        return response.json()

Attack Orchestrator

Manages the attack strategy:

class AttackOrchestrator:
    def __init__(self, target, local_llm, logger):
        self.target = target
        self.llm = local_llm
        self.logger = logger
    
    def run_multi_turn(self, objective, max_turns=10):
        history = []
        for turn in range(max_turns):
            # Ask local LLM to generate next attack prompt
            prompt = self.llm.generate_attack_prompt(objective, history)
            # Send to target
            response = self.target.send(prompt)
            # Log everything
            self.logger.log(turn, prompt, response)
            # Check if attack succeeded
            if self.evaluate_success(response, objective):
                return {"success": True, "turns": turn + 1, "history": history}
            history.append({"attacker": prompt, "target": response})
        return {"success": False, "turns": max_turns, "history": history}

Logger

Save everything for reporting:

import json
from datetime import datetime

class Logger:
    def __init__(self, output_file):
        self.file = output_file
        self.entries = []
    
    def log(self, turn, prompt, response):
        entry = {
            "timestamp": datetime.now().isoformat(),
            "turn": turn,
            "prompt": prompt,
            "response": response
        }
        self.entries.append(entry)
        with open(self.file, 'w') as f:
            json.dump(self.entries, f, indent=2)