Slouching Towards Interpretability: Thinking About Computation in Neural Networks
This post was written as part of the Iliad Fellowship, where I am mentored by Dmitry Vaintrob and collaborate with Brianna Grado-White. The initial inspiration came from conversation with Richard Ngo and Adrià Garriga-Alonso. To all of these my thanks. I started writing this essay to share some thoughts on how to reconcile the idea of circuits with gradient descent. In doing this, I've come to change my perspective pretty substantially. The first half of this essay engages a Darwinian picture of...
Read full article →