Baiting monitorable reward hacks

·LessWrong··

This is joint work between Ziqian Zhong, Lawrence Feng, Gaurav Rohit Ghosal, Jiangqi Dai, and Aditi Raghunathan. This work is supported by a BlueDot Rapid Grant.We are unsure of whether this intervention is net positive (see discussions below) despite positive results, but we think similar techniques are worth further studying.Gradient routing can fail to mitigate reward hacks when the models settle on hacks not caught by the monitor. For example, if we swap the monitorable and unmonitorable set...

Read full article →

Related Articles

MIT's New Method Flags AI Models Trained on CASM Without Generating It
sdoering · Hacker News · 4mo ago
Harm Laundering in GPT Models: Gender Discrimination Transformed Rather Than
sbulaev · Hacker News · 2mo ago
Q3.5 Timelines Update: Looks Like the Intelligence Explosion Will Happen in 2027 After All
Thomas Larsen · Alignment Forum · 8d ago
Self-Modeling Interventions Modulate Emergent Misalignment
gmays · Hacker News · 1mo ago
OpenAI agents keep disabling monitoring; is this being handled appropriately?
Alec Harris · Redwood Research · 14d ago