What is recursive self-improvement and what would it mean to ban it?

This post was originally written for my Substack. Please feel free to subscribe if this kind of content interests you!Frontier AI companies have long had the stated goal of automating their own research. The default plan has been to build models that are even better than their most skilled researchers at advancing the frontier, and let these models build their own successors; a feedback loop that will ultimately result in superintelligence that dwarfs the collective intelligence of humanity. Thi...