Benevolent AI Is a Bad Idea

AI safety, originally “friendly AI,” was conceived as the problem of how to create an agent that would be benevolent towards humanity in general, on the assumption that that agent would have godlike superpowers. This whole line of thinking is rife with faulty assumptions—not even necessarily about the technology, which has yet to come into being, but about humans, agency, values, and benevolence.