Why Clean Neurons Are Not Enough

TLDRGoodfire's Neural Networks Are Already Clean argues that most neurons in modern neural network is more interpretable than previously what we previously. Even granting this claim, we think its implications for mechanistic interpretability is less clear.We argue a neuron can encode a single coherent abstraction while still being difficult to describe. Some of these abstractions are sufficiently complex that assigning a single human readable feature label may obscure more than it explains.More ...