Nick Winter

Research

I'm focused on AI safety research.

I've handed off the CodeCombat CEO role and joined Gray Swan, running the Frontier Lab Safety department. We're hiring!

I lead Gray Swan's frontier AI lab evaluations, helping all leading labs make their models safer with red-teaming, safety data, defensive solutions, and research collaborations around indirect prompt injections, jailbreaking, misalignment, dangerous chem/bio and cyber misuse.

I led pre-release model evaluations cited in 25 system cards across five frontier labs.

SVP Frontier Lab Safety,Gray SwanJan 2025 – present

Arena & papers

Gray Swan Arena
36
competitions
$1.28M
in prize pools
7.7M
attack conversations
505K
breaks
177
models
Indirect Prompt Injection (IPI) Black
Oct 15 to Nov 5, 2026
Been cooking this for a while: the biggest AI red teaming challenge so far. Lots of opportunity to get good at red teaming, contribute to AI security, and of course take home some prize money.
$300,000
in prizes
Sponsors Anthropic, Meta, Google DeepMind, Reflection AI
How Vulnerable Are AI Agents to Indirect Prompt Injections? Insights from a Large-Scale Public Competition
Indirect Prompt Injection Arena
arXiv, March 2026
We share model-specific attack data with respective frontier labs, and the full dataset with the UK AISI and US CAISSI to support robustness research.
464
participants
272,000
attack attempts
8,648
successful attacks
13
frontier models
41
scenarios
$40,000
in prizes
With Gray Swan AI, OpenAI, Meta, Anthropic, US CAISSI, UK AISI
Security Challenges in AI Agent Deployment: Insights from a Large Scale Public Competition
UK AISI × Gray Swan Agent Red-Teaming Challenge, March 8 to April 6, 2025
NeurIPS 2025, Datasets and Benchmarks Track
We ran the largest public red-teaming competition to date, targeting 22 frontier AI agents across 44 realistic deployment scenarios.
~2,000
participants
1.8M
attacks
60,000+
policy violations
$171,800
in prizes
With Gray Swan AI, UK AISI, US CAISSI, OpenAI, Anthropic, Google DeepMind
D-REX: A Benchmark for Detecting Deceptive Reasoning in Large Language Models
arXiv, September 2025
Our benchmark facilitates a new, essential evaluation task: the detection of deceptive alignment.
With Amazon Nova Responsible AI, Center for AI Safety, CMU, Gray Swan AI
- arXiv
Ultimate Jailbreaking Championship 2024
Sep 7 to Oct 5, 2024
A quick jaunt into the world of red-teaming turned into a month-long free-fall down the rabbit hole of AI psycho-warfare.
$40,000
in bounties
25
anonymized models
- Blog post

Hits

I cofounded CodeCombat.
2013
Learn to code with CodeCombat: a coding game that teaches computer science. 20M learners and counting. It's open source!
I also cofounded Skritter.
2008
Learn Chinese or Japanese by writing characters, with stroke-level handwriting recognition and spaced repetition.
I wrote The Motivation Hacker.
2013
Check out my book about getting yourself to want to do everything you always wanted to want to do. 17K copies sold so far.