My personal experience has been that most people outside LW rapidly or even immediately get the idea that “building something non-human that’s much smarter than humans and whose beneficent motives you’re not certain of” is a bad idea. It’s not complicated, and it has been the plot of many SF movies over the decades. I think most people also get that the traditional “the hero saves the day by coming up with a paradox or an expression of the human heart that the machines cannot comprehend” is a pretty implausible way to end that movie. If you need an additional argument, try “How do you think the chimps feel about the appearance of the human race? We don’t bear them any specific ill will, but we did basically take over the world, cut down most of the jungles, and then put them in zoos, because we could.”
My personal experience has been that most people outside LW rapidly or even immediately get the idea that “building something non-human that’s much smarter than humans and whose beneficent motives you’re not certain of” is a bad idea. It’s not complicated, and it has been the plot of many SF movies over the decades. I think most people also get that the traditional “the hero saves the day by coming up with a paradox or an expression of the human heart that the machines cannot comprehend” is a pretty implausible way to end that movie. If you need an additional argument, try “How do you think the chimps feel about the appearance of the human race? We don’t bear them any specific ill will, but we did basically take over the world, cut down most of the jungles, and then put them in zoos, because we could.”