Also, discussions of caution may increase the number of “desirable organizations” working to develop AI. In terms of your model, such discussion could turn a black-hat organization into a smiley-faced one.
That seems like the (dubious) “engineers are incompetent and a bug takes over the world” scenario.
I think a much more obvious concern is where the “engineers successfully build the machine to do what it is told” scenario—where the machine helps its builders and sponsors—but all the other humans in the world—not so much.
That seems like the (dubious) “engineers are incompetent and a bug takes over the world” scenario.
I think a much more obvious concern is where the “engineers successfully build the machine to do what it is told” scenario—where the machine helps its builders and sponsors—but all the other humans in the world—not so much.