Why I'm doing the Susan Calvin Project

tl;dr — The evals ecosystem needs to be complemented with real-world monitoring. The AI labs can and should monitor their own traffic, but we also need an independent voice that keeps labs accountable and monitors open models. At the Susan Calvin Project, we aim to detect AI misbehaviors and incidents in the wild, and collect evidence for the (mis)alignment of existing AIs. Agentic AI tools are integrated into more aspects of our work and personal life, and models continue to become more capable...