Visualize Cyclical Structure in Llama Model

SummaryResearch increasingly shows that various geometric structures emerge in the activation and behavior spaces of large language models. These structures are fascinating to me, and I find it worth exploring what structures we can find for different concepts in LLM’s. I made a tool allowing anyone to generate and visualize 3d structures for some arbitrary concept. The tool currently has support for cyclical and smooth structures, extracting activations at any given layer in the Llama-3-8b mode...