Exploring the data landscape via metagradient descent

This project was done as part of the Iliad Fellowship. L.J. did this work while at EleutherAI. From functions to data That we can learn efficiently from our environment at all tells us something about (1) the world and (2) ourselves: (1) Regularities in the world are simple enough to be picked up from limited experience. (2) Our inductive biases favour precisely those regularities. Learning, in this case, is effective because the two are matched. Considering a neural network with a fixed archite...