Intrinsic Drives and Extrinsic Misuse: Two Intertwined Risks of AI

Given their advanced capabilities, future AI systems could pose significant risks to society. Some of this risk stems from humans using AI systems for bad ends (misuse), while some stems from the difficulty of controlling AI systems “even if we wanted to” (misalignment). We can analogize both of these with existing risks. For misuse, we can consider the example of nuclear weapons, where the mass-production of hydrogen bombs created an existentially precarious situation. If the world’s arsenal of...