Opening up Attention: A QK Transcoder Dashboard for Feature Pairs

Multi Head Attention in Transformers continues to be an elusive topic in mechanistic interpretability. One way of trying to decompose MHA into interpretable parts is applying SDL (Sparse Dictionary Learning) methods to the attention mechanism. However, despite the progress that has been made, we still don't have a firm grasp on attention patterns and their formation.Additionally, there is an absence of comprehensive dashboard views for the attention mechanism and features present therein. Featur...