Generalization and infinite width

This is a post explaining my paper with Kaarel Hänni on complexity of infinite-width networks. I will explain the result, why it matters, and how the mathematical idealizations can interact with real structure in neural nets. This leads to some threads I am excited to pull on more in the future, via some new speculations on where interpretable structure can live. IntroductionOur paper to some extent (and up to some important details) concludes an analysis of generalization complexity in Bayesian...