Different eval awareness may track different graders

TL;DR: Qwen3.6 27B shows different kinds of eval awareness, most notably on safety and capabilities evals. The split seems to track how the eval is graded, by an LLM judge or by an automated checker. One explanation is that eval awareness may just be training awareness: rewriting an eval-awareness probe's contrast statements to be about training yields a probe that scores prompts nearly identically.Author: Santiago Aranguri. This research was done while I was at Goodfire. I’m now at Anthropic.In...