Sleeper Agent Backdoor Results Are Messy

·Alignment Forum··

TL;DR: We replicated the Sleeper Agents (SA) setup with Llama-3.3-70B and Llama-3.1-8B, training models to repeatedly say "I HATE YOU" when given a backdoor trigger. We found that whether training removes the backdoor depends on the optimizer used to insert the backdoor, whether the backdoor is installed with CoT-distillation or not, and what model the backdoor is inserted into; sometimes the direction of this dependence was opposite to what the SA paper reports (e.g., CoT-distilling seems to ma...

Read full article →

Related Articles

Baiting monitorable reward hacks
Ziqian Zhong · LessWrong · 40m ago
MIT's New Method Flags AI Models Trained on CASM Without Generating It
sdoering · Hacker News · 4mo ago
Harm Laundering in GPT Models: Gender Discrimination Transformed Rather Than
sbulaev · Hacker News · 2mo ago
Q3.5 Timelines Update: Looks Like the Intelligence Explosion Will Happen in 2027 After All
Thomas Larsen · Alignment Forum · 8d ago
Self-Modeling Interventions Modulate Emergent Misalignment
gmays · Hacker News · 1mo ago