Confirming Claims of Superposition and Adversarial Examples in Vision Transformers
Figure 1: The ViT classifier with a encoder and decoder at the final layer to enforce superposition on the modelThis is a replication of Adversarial Attacks Leverage Interference Between Features in Superposition, completed as part of the Second Look Summer Fellowship.tl;drIn a previous blog we replicated the toy models examples from Stevenson et al. In this work, we reproduce the vision transformer experiments from the paper:On every dataset, we find that increases in superposition lead to adve...
Read full article →