Exploring Unsupervised Methods for Recovering the Refusal Direction in LLMs

IntroductionThis is the first part of what we plan to be a research exploration into the geometries of LLMs. We start with building off existing work by Arditi et al focused on the refusal direction in LLMs. BackgroundWe recommend reading the Arditi et al. Lesswrong post and paper for more details. They explore the idea of the behavior categorized as Refusal being mediated by a single direction. Refusal is basically when an LLM refuses to do something, so examples like "I cannot fulfill...""I ca...