An Induction Head in Disguise: Chasing Grammar in a Character-Level Transformer

Confident in the measurements — copying score, OV logits, attention patterns. More exploratory on the interpretations of what the head "knows," two of which I ended up disconfirming myself. My first mechanistic interpretability post; corrections welcome.Full analysis and code: github/Ameya-bit/QuotesByNicheI trained a small character level transformer on Nietzsche texts, and now I interpret on it. Numbers can be flattering, but it's the control that breaks, diverts, or makes a conclusion. In try...