Measuring Reward-Seeking via Contrastive Belief Updates

This is a linkpost for https://rewardseeking.ai/Machine learning models can produce the right outputs for the wrong reasons. Famous examples include a reinforcement learning agent that, rewarded for collecting a coin always placed at the right end of the level, learns to run rightward rather than to seek the coin itself (Langosco et al., 2022; Shah et al., 2022). Another example is a pneumonia classifier that learns to recognize which hospital took an X-ray rather than features of the disease (Z...