t seems to me like probably, the answer to the question of how to make AIs benevolent isn’t vastly more complicated than the answer of how to make them smart.
Doesn’t pretty much everything on the complexity of human values point against this?
The problem with talking about how X is complex is that it leaves the question, “complex relative to what?” It certainly looks complex relative to attempts to various attempts to state all of moral theory in a sentence or three. But I tend to think the software for intelligence would also have to be much more complex than that.
Creating something very complicated isn’t necessarily very hard if you can figure out how to specify what you want indirectly, invoking tools that would do it for you. (What we know is that it doesn’t happen on its own (by symmetry with alternative outcomes) and that so far it’s not clear how to do it.)
Doesn’t pretty much everything on the complexity of human values point against this?
The problem with talking about how X is complex is that it leaves the question, “complex relative to what?” It certainly looks complex relative to attempts to various attempts to state all of moral theory in a sentence or three. But I tend to think the software for intelligence would also have to be much more complex than that.
Creating something very complicated isn’t necessarily very hard if you can figure out how to specify what you want indirectly, invoking tools that would do it for you. (What we know is that it doesn’t happen on its own (by symmetry with alternative outcomes) and that so far it’s not clear how to do it.)