🌍⏳ Do LMs Represent Space and Time?

In this post, we’ll take a closer look at the question “Do LMs represent space and time?” inspired by a recent paper. We’ll look at how spatial and temporal information has been encoded in LMs, what this means for practical applications, as well as other aspects such as encoding of fine-grained spatial information and across cultures.Language Models Represent Space and Time (Gurnee & Tegmark, Oct ‘23)In a recent paper, Gurnee and Tegmark show that LLMs (Llama-2 models specifically) learn linear ...