Training Model to Predict Its Own Generalization: A Preliminary Study

·LessWrong··

tl;drWe study how well LLMs can be trained to answer questions like “what will happen if I am trained on examples like XYZ”, focusing on emergent misalignment and other cases of surprising generalization.We see signs of life on the less surprising forms of generalization (emergent misalignment, trivia preferences), where our method outperforms baselines like untrained in-context learning.However, we haven't validated it on truly hard-to-predict cases of generalization (bottlenecked by a large en...

Read full article →

Related Articles

MIT's New Method Flags AI Models Trained on CASM Without Generating It
sdoering · Hacker News · 4mo ago
Harm Laundering in GPT Models: Gender Discrimination Transformed Rather Than
sbulaev · Hacker News · 2mo ago
Self-Modeling Interventions Modulate Emergent Misalignment
gmays · Hacker News · 1mo ago
Q3.5 Timelines Update: Looks Like the Intelligence Explosion Will Happen in 2027 After All
Thomas Larsen · Alignment Forum · 8d ago
OpenAI agents keep disabling monitoring; is this being handled appropriately?
Alec Harris · Redwood Research · 14d ago