A superintelligent computer designed to win at chess will keep trying to win at chess, ignoring any other goals along the way. It doesn’t matter whether it’s a million times smarter than Einstein, it’s not going to start wanting to fight for freedom just because humans like that kind of thing any more than it’s going to start wanting to have sex with pretty actors and actresses just because humans like that kind of thing. It’s just going to be really, really good at winning at chess.
-Yvain