What is recursive self-improvement, and what would it mean to ban it? by sarahhw

This post was origi­nally writ­ten for my Sub­stack. Please feel free to sub­scribe if this kind of con­tent in­ter­ests you!Fron­tier AI com­pa­nies have long had the stated goal of au­tomat­ing their own re­search. The de­fault plan has been to build mod­els that are even bet­ter than their most skil­led re­searchers at ad­vanc­ing the fron­tier, and let these mod­els build their own suc­ces­sors; a feed­back loop that will ul­ti­mately re­sult in su­per­in­tel­li­gence that dwarfs the col­lec...