CLT Features Support Manifold Steering, but Hold Only Partial Steering Signal
tl;dr I continue work to extend the Goodfire lab's cyclical manifold methods to Anthropic's pre-trained CLT feature space on Gemma-2-2B. I find that manifold steering produces the expected cyclical transition between days of the week in both activation space and CLT feature space, but that CLT features provide a weaker steering signal, suggesting that the MLP sublayers in which they originate only hold a portion of the full day-of-the-week signal. Colab notebook implementation can be found here....
Read full article →