The Misguided Panic About Superintelligence

A recent Persuasion article by Andrea Miotti argues that “We Need an International Treaty to Ban Superintelligence.” His argument—like that of many others in what we might call doomer circles—is that large-scale Generative AI systems, like ChatGPT and Claude, are fast approaching a sharp transition from their current trajectory to runaway superintelligence, at which point they may escape human control, reprogram themselves to adopt new, self-directed goals, and possibly wipe out humanity entirely.