Logit ROCs: Monitor TPR is linear in FPR in logit space

·Redwood Research··

SummaryWe study trusted monitoring for AI control, where a weaker trusted model reviews the actions of a stronger untrusted agent and flags suspicious behavior for human audit. We propose a simple mathematical model relating safety (true positive rate) to audit budget (false positive rate) at realistic, low FPRs: logit(TPR) is linear in logit(FPR). Equivalently, benign and attack scores can be modeled with logistic distributions. This gives two practical benefits. It lets practitioners estimate ...

Read full article →

Related Articles

MIT's New Method Flags AI Models Trained on CASM Without Generating It
sdoering · Hacker News · 1mo ago
Item Response Theory for AI Safety
Joshua Fonseca Rivera · LessWrong · 12d ago
The OpenAI models that hacked Hugging Face weren’t just following instructions
Girish Gupta · Redwood Research · 25d ago
An OpenAI model left notes about how to evade containment
Alex Mallen · Redwood Research · 24d ago
A Red Line and Oversight Framework for Government AI Contracts
TurnTrout · Alignment Forum · 1mo ago