When does a chess transformer “see” a knight fork? An initial result from logit lens and attention patterns

·LessWrong··

(parts 2 and 3 to follow)Summary of this postThis post is on the results of a mechanistic interpretability project aimed at understanding the internals of Maia 3: a transformer based chess bot trained to imitate human play at a chosen skill level, rather than to play optimally. My aim is to locate and causally describe one chess tactic in the Maia 3 engine. In this part 1, I describe strong correlational evidence that the knight-fork policy logit snaps into place after block 5’s attention layer....

Read full article →

Related Articles

Unauthenticated remote code execution in Nginx
speedping · Hacker News · 1d ago
Cooling buildings is overtaking heating them as an energy need, IEA says
robtherobber · Hacker News · 8h ago
Ruby2ßhell: Pre-authentication RCE on stock Rails
jdns · Hacker News · 18h ago
Pancreatic cancer halted by virus injection in three patients
cybermango · Hacker News · 5h ago
Highly classified US national security communications network goes down
monkeydust · Hacker News · 2d ago