To create a superhuman AI driver, you ‘just’ need to create a realistic VR driving sim and then train a ULM in that world (better training and the simple power of selective copying leads to superhuman driving capability).
So to create benevolent AGI, we should think about how to create virtual worlds with the right structure, how to educate minds in those worlds, and how to safely evaluate the results.
There is some interesting overlap between these ideas and Eric Drexler’s recent proposal. (Previously discussed on LessWrong here)
Separating learning capacity from domain knowledge is kind of automatic in a ULM approach. There is nothing inherently dangerous about the learning mechanisms itself—it’s the knowledge that is potentially dangerous. I have butted heads with LW on that point for 4 to 5 years.
The knowledge management idea is the essence of the VR sandbox approach, but I also imagine separating out value systems/priors to some degree for independent testing. Overall Drexler’s proposal (from reading the abstract and skimming) seems to be very much in line with my views.
Safety considerations would go into design at all levels, from designing the VR world itself to the brain architecture to the education/training programs.
In regards to modularity: large ANN systems are already modular, brains are modular, and brain-style AGI approaches are modular. It’s just sort of assumed. It’s a new consideration for perhaps the formal/math/AIXI/MIRI cluster, but only because they haven’t put as much thought into practical architecture.
There is some interesting overlap between these ideas and Eric Drexler’s recent proposal. (Previously discussed on LessWrong here)
Cool—hadn’t read that yet.
Separating learning capacity from domain knowledge is kind of automatic in a ULM approach. There is nothing inherently dangerous about the learning mechanisms itself—it’s the knowledge that is potentially dangerous. I have butted heads with LW on that point for 4 to 5 years.
The knowledge management idea is the essence of the VR sandbox approach, but I also imagine separating out value systems/priors to some degree for independent testing. Overall Drexler’s proposal (from reading the abstract and skimming) seems to be very much in line with my views.
Safety considerations would go into design at all levels, from designing the VR world itself to the brain architecture to the education/training programs.
In regards to modularity: large ANN systems are already modular, brains are modular, and brain-style AGI approaches are modular. It’s just sort of assumed. It’s a new consideration for perhaps the formal/math/AIXI/MIRI cluster, but only because they haven’t put as much thought into practical architecture.