The potentially deadly threat of AI output-optimization

“You do not rise to the level of your goals. You fall to the level of your systems.”— James Clear, Atomic Habits“We cannot improve results ‘directly.’ It can only be achieved by improving those processes [...] that produce results.”— Orest FiumeThe much discussed “P(doom)” question asks about the likelihoods we’d ascribe to the most lethal possible futures of AI. What happens when we invert this question? It transforms into the presumably less urgent, though I’d say still important, form: What a...