Is there even a ground-truth for LLMs’ internal representations?

[This is an introductory blog for the paper Laguerre Geometry for Interpreting Large Language Models and the GitHub repository Geometric Lens.]LLM Lens: What does an internal vector mean?Anthropic's recent paper on the "J-Lens" (Jacobian Lens) has revived interest in reading the "thoughts" inside Large Language Models. The idea of placing a “lens” at an LLM's hidden layers isn't new. It dates back to the Logit Lens, and has since evolved into a family of variants, including Tuned Lens and Patchs...