RL Reward-Hack Auditing for Chinese AI Safety
TL;DR:Access to AI safety talent & resources, particularly for labs during training, is sorely lacking in China. We created one of the largest AI safety communities in China, and plan to next start with auditing reward-hacks of unreliable upstream RL environments that domestic data vendors provide to Chinese frontier labs (which induces reward hacking and a whole host of dangerous downstream tendencies).RL tasks are notoriously prone to exploitation by reward-seeking agents (see examples here, h...
Read full article →