Optimiser Choice Can Amplify or Suppress Emergent Misalignment

·LessWrong··

This is a linkpost for https://arxiv.org/abs/2606.31591. Work done with Patrick Leask and Lev McKinney during the Astra Fellowship.TL;DR: Optimiser choice strongly influences emergent misalignment, while model size and family seem to barely matter. Optimisers that concentrate the LoRA update into fewer directions degrade alignment more, but regularising towards a flatter spectrum can mitigate this and improve alignment. There are some follow-up directions I (Jason) would be happy to advise or me...

Read full article →

Related Articles

There's no reason for software to be slow anymore
Jach · Hacker News · 1d ago
JIT Compiling Code in 5μs
zX41ZdbW · Hacker News · 8h ago
MartyPC is a cross-platform emulator of early PCs written in Rust
boilerupnc · Hacker News · 10h ago
hdiutil is deprecated in macOS 27 Golden Gate
zdw · Hacker News · 19h ago
Malicious Rust crate Arrayref runs a build-time payload
abhisek · Hacker News · 3d ago