How to Train Your Robot

The enormous advance in AI capabilities over the last several years has in large part been driven by the availability of vast amounts of training data. Gwern’s influential 2020 essay “The Scaling Hypothesis,” written after the release of GPT-3, argues that recent advances in AI have been the result of scale: bigger models, bigger datasets, and more compute spent training them. Gwern writes:The scaling hypothesis regards the blessings of scale as the secret of AGI: intelligence is ‘just’ simple n...