Optimiser Choice Can Amplify or Suppress Emergent Misalignment

·LessWrong··

This is a linkpost for https://arxiv.org/abs/2606.31591. Work done with Patrick Leask and Lev McKinney during the Astra Fellowship.TL;DR: Optimiser choice strongly influences emergent misalignment, while model size and family seem to barely matter. Optimisers that concentrate the LoRA update into fewer directions degrade alignment more, but regularising towards a flatter spectrum can mitigate this and improve alignment. There are some follow-up directions I (Jason) would be happy to advise or me...

Read full article →

Related Articles

Postgres rewritten in Rust, now passing 100% of the Postgres regression tests
SweetSoftPillow · Hacker News · 6h ago
John Deere owners will get the right to repair equipment under FTC settlement
djoldman · Hacker News · 12h ago
The classifiers Anthropic puts in front of Fable are too zealous
karrot-kake · Hacker News · 15h ago
Spider venom kills varroa mites without harming honeybees
Jedd · Hacker News · 7h ago
Decoding the obfuscated bash script on a Uniqlo t-shirt
speerer · Hacker News · 1d ago