CLT Features Sharpen the Cyclical Day-of-Week Manifold in Gemma-2-2b

tl;dr: I reproduced the Goodfire lab's cyclical manifold result using days-of-the-week on Gemma-2-2b and found that Anthropic's pre-trained CLT features produce an even cleaner cyclical manifold with more distinct features than the raw activations or output probabilities. Colab notebook implementation can be found here.IntroductionIn May of this year, Wurgaft et al. (2026) reproduced a known result that days-of-the-week (and other semantically cyclical and sequential relationships) are represent...