Rodney Brooks talks about Evil AI and mentions MIRI [LINK]
Rodney Brooks says that “evil” AI is not a big problem:
http://www.rethinkrobotics.com/artificial-intelligence-tool-threat/
Rodney Brooks says that “evil” AI is not a big problem:
http://www.rethinkrobotics.com/artificial-intelligence-tool-threat/
The MIRI mention:
Do you feel that is a fair summary of your report?
Yeah.
He is, perhaps, a little glib. And I would not dismiss some kind of left-field breakthrough in the next 25 years that brings us close to AI.
But other than that I agree with most of his statements. We are fundamental leaps away from understanding how to create strong AI. Research on safety is probably mostly premature. Worrying about existing projects, like Googles’, having the capacity to be dangerous is nonsensical.
I place most of my probability weighting on far-future AI too, but I would not endorse Brooks’s call to relax. There is a lot of work to be done on safety, and the chances of successfully engineering safety go up if work starts early. Granted, much of that work needs to wait until it is clearer which approaches to AGI are promising. But not all.
Well, he’s right that intentionally evil AI is highly unlikely to be created:
which happens to be the exact reason why Friendly AI is difficult. He doesn’t directly address things that don’t care about humans, like paperclip maximizers, but some of his arguments can be applied to them.
He’s totally right that AGI with intentionality is an extremely difficult problem. We haven’t created anything that is even close to practically approximating Solomonoff induction across a variety of situations, and Solomonoff induction is insufficient for the kind of intentionality you would need to build something that cares about universe states while being able to model the universe in a flexible manner. But, you can throw more computation power at a lot of problems to get better solutions, and I expect approximate Solomonoff induction to become practical in limited ways as computation power increases and moderate algorithmic improvements are made. This is true partially because greater computation power allows one to search for better algorithms.
I do agree with him that human-level AGI within the next few decades is unlikely and that significantly slowing down AI research is probably not a good idea right now.
I think the key points (or misunderstandings) of the post can be seen in these quotes:
and
Which seem to indicate that Brooks doesn’t look past ‘linear’ scaling and sees composition effects as far away.
Apparently he extrapolates his own specialy into the future.