He seems to be pretty optimistic about an artificial intelligence architecture requiring lots of custom-crafted crunchy bits to go from being able to autonomously clean and fix your apartment, buy you groceries, make you lunch, drive you around town and figure out it needs to save your life and do so when you get a cardiac arrest to being able to do significant runaway learning and reasoning that will have serious and cumulative unintended consequences. And per Omohundro’s Basic AI Drives paper, needing to have efficient real-world intelligence might count as a potentially dangerous motivation on its own, so not giving the things a goal system cribbed from humans won’t help much.
He seems to be pretty optimistic about an artificial intelligence architecture requiring lots of custom-crafted crunchy bits to go from being able to autonomously clean and fix your apartment, buy you groceries, make you lunch, drive you around town and figure out it needs to save your life and do so when you get a cardiac arrest to being able to do significant runaway learning and reasoning that will have serious and cumulative unintended consequences. And per Omohundro’s Basic AI Drives paper, needing to have efficient real-world intelligence might count as a potentially dangerous motivation on its own, so not giving the things a goal system cribbed from humans won’t help much.