Are there other advances in computer science that might show up within the next twenty years, that would make friendly-AI much less interesting?
Well the general counter is not advances in computer science, but rather the idea that a rogue agent getting much more powerful that other public players that integrate into the rest of society is unlikely.
Today machine intelligence integrates into people’s mobile phones, desktop computers. It knows about privacy. It knows when to be unobtrusive. It knows what people like. That process seems likely to continue for quite a while, with machines absorbing human values along the way. It is easier to cooperate with humans than to recycle their atoms, especially when they have direct access to mature advanced molecular nanotechnology bodies, and you don’t.
So, we will likely initally see a global advance, that is integrated into human society, and which involves machines absorbing human values, so as to better give them what they want. That doesn’t address longer-term questions—but it does make the scenario of an uncaring machine intelligence arising first and trashing the planet seem relatively unlikely. Instead, superhuman machine intelligence will arise out of subhuman machine intelligence—which will by that point most likely have an extended history of cooperating with humans and respecting their values.
Well the general counter is not advances in computer science, but rather the idea that a rogue agent getting much more powerful that other public players that integrate into the rest of society is unlikely.
Today machine intelligence integrates into people’s mobile phones, desktop computers. It knows about privacy. It knows when to be unobtrusive. It knows what people like. That process seems likely to continue for quite a while, with machines absorbing human values along the way. It is easier to cooperate with humans than to recycle their atoms, especially when they have direct access to mature advanced molecular nanotechnology bodies, and you don’t.
So, we will likely initally see a global advance, that is integrated into human society, and which involves machines absorbing human values, so as to better give them what they want. That doesn’t address longer-term questions—but it does make the scenario of an uncaring machine intelligence arising first and trashing the planet seem relatively unlikely. Instead, superhuman machine intelligence will arise out of subhuman machine intelligence—which will by that point most likely have an extended history of cooperating with humans and respecting their values.