Optimiser Choice Can Amplify or Suppress Emergent Misalignment

·LessWrong··

This is a linkpost for https://arxiv.org/abs/2606.31591. Work done with Patrick Leask and Lev McKinney during the Astra Fellowship.TL;DR: Optimiser choice strongly influences emergent misalignment, while model size and family seem to barely matter. Optimisers that concentrate the LoRA update into fewer directions degrade alignment more, but regularising towards a flatter spectrum can mitigate this and improve alignment. There are some follow-up directions I (Jason) would be happy to advise or me...

Read full article →

Related Articles

Mistral Large 4
Philpax · Hacker News · 1d ago
Shipping JPEG XL in Chrome
AshleysBrain · Hacker News · 5h ago
JetBrains reported a net financial loss first time in its tracked history
thw_9a83c · Hacker News · 1d ago
AnyPS5: Port PS5 binaries to PC without emulation (87% system libraries mapped)
Fe2O3 · Hacker News · 17h ago
OpenTPU – An open-source AI accelerator, developed by AI
fsbonetto · Hacker News · 1d ago