Organisation
Gray Swan AI
Red-teaming of frontier models and jailbreak-resistance defences (Cygnal, Shade), plus a crowdsourced adversarial-testing arena.
Gray Swan AI is a US safety-and-security company that red-teams frontier models and builds defences against jailbreaks and prompt injection. It was founded in 2024 by Carnegie Mellon researchers Matt Fredrikson and Zico Kolter, together with Andy Zou, who had developed some of the field's foundational techniques for attacking and hardening large language models. The company runs an "Arena" that crowdsources adversarial testing from thousands of hackers and sells products it calls Cygnal (input/output filtering and monitoring) and Shade (deployment pressure-testing); it has said it works with labs including OpenAI and Anthropic. It raised a $40m Series A in 2025 led by Madrona and Wing Venture Capital.
- Category
- Safety & alignment research
- Founded
- 2024
- HQ
- Pittsburgh, US
- Funding
- $40m Series A (2025), led by Madrona and Wing Venture Capital
- Key people
- Matt Fredrikson, Zico Kolter, Andy Zou