LLMs can improve CoT Controllability via Reward Laundering

This is a two week sprint project done as part of the Redwood Research internship. afterless was the main IC, james.lucassen supervised.TL;DR – Earlier work found that Qwen 3.5 9B can manipulate its RL process on one task to self-improve on an unrelated/unrewarded task. We find that Qwen 3.8 27B can use the same capability to improve on a CoT controllability task and generalize to other unseen CoT controllability tasks, while retaining/regaining main task performance despite initial degradation....