Self-Improving Robots and the Importance of Data

There is a popular idea that, with 10,000 hours of experience, a person could achieve mastery at whatever task they set themselves to. A bit of back-of-the-envelope calculation tells us that so far, our machine learning models are quite a bit more data-hungry. For example, GPT-3 is trained on over 10 terabytes of data, which corresponds to some billions of pages of text that, at average human reading speed, would take hundreds of millions of hours to read (about a hundred lifetimes). State-of-th...