Small Models Have A Global Workspace

tl;dr: whether a model has one tracks modern pretraining, not parameter count, and not instruction-tuning.Here's the intuition almost everyone has about a "global workspace" in a language model. It's a big-model thing. Something that only switches on once you have tens of billions of parameters, or something the instruction-tuning bolts on afterward to make the model act like it has an inner life. Both of those guesses are wrong, and I can show you why on a desktop that cost me nothing to run.Sm...