Eval Awareness as Eval-type Awareness
The way eval awareness is discussed in the AI safety discourse, either here or on X, can be misleading. Before today, I think I had the wrong mental model of why eval awareness was a problem, and I am writing here in case anyone was making the same errors in their thinking, as well a present a working theory. As laid out in "A Framework for Eval Awareness" the problem is that models who know they are being evaluated will behave differently than they would in deployment. This raises a huge proble...
Read full article →