AI Security & Safety
Attacks on AI systems and how to defend them: prompt injection, red teaming, and the frameworks security teams use.
Advanced
Once the basics are comfortable.
garak: The LLM Vulnerability Scanner
Scan a language model for weak spots such as prompt injection, jailbreaks, data leakage and toxicity, much like a network scanner checks a server.
Sign up to see the source
Google Secure AI Framework (SAIF)
••••'s framework of practical controls for securing AI systems across data, models, deployment and monitoring, with a risk map to match.
Sign up to see the source
Microsoft AI Red Team
How •••• tests AI systems by attacking them first: the approach, the lessons, and the tools and guidance it shares with others.
Sign up to see the source
NIST: Adversarial Machine Learning Taxonomy
A government standard that names and organises attacks on AI (evasion, poisoning, privacy and abuse attacks) and the defences for each. Useful as a reference list.
Sign up to see the source
PyRIT: Python Risk Identification Tool for Generative AI
An open-source toolkit for red-teaming generative AI: automate attack prompts against a model, score the answers and keep records of what you tried.
Sign up to see the source