I think it is possible to make a machine that does well at all these tasks, not because it has a separate hardcoded subsection for each task, but because there are simple general principles, like occams razor and updating probability distributions, that can be applied to all of them.
The existence of the human brain, which does pretty well at a wide variety of tasks, despite those tasks not being hard coded in by evolution, provides some evidence for this.
AIXI is a theoretical AI that brute force simulates everything. It should do extremely well on all of these tasks. Do you agree that if we had infinite compute, AIXI would be very good at all tasks, including hacking its reward channel.
Do you agree that there is nothing magical about human brains?
Like many philosophical arguments against superintelligence, is doesn’t make clear where the intelligence stops. Can a single piece of software be at least 2500 dan at chess, and be able to drive a car for a million miles without accident? Can it do that and also prove the reinmann hypothesis and making a billion dollar startup? Looking at compute or parameters, you might be able to say that no AI could achieve all that with less than X flops. I have no idea how you would find X. But at least those are clear unambiguous technical predictions.
Take a list of tasks such as
Winning a chess game
Building a mars rover
Writing a good novel
....
I think it is possible to make a machine that does well at all these tasks, not because it has a separate hardcoded subsection for each task, but because there are simple general principles, like occams razor and updating probability distributions, that can be applied to all of them.
The existence of the human brain, which does pretty well at a wide variety of tasks, despite those tasks not being hard coded in by evolution, provides some evidence for this.
AIXI is a theoretical AI that brute force simulates everything. It should do extremely well on all of these tasks. Do you agree that if we had infinite compute, AIXI would be very good at all tasks, including hacking its reward channel.
Do you agree that there is nothing magical about human brains?
Like many philosophical arguments against superintelligence, is doesn’t make clear where the intelligence stops. Can a single piece of software be at least 2500 dan at chess, and be able to drive a car for a million miles without accident? Can it do that and also prove the reinmann hypothesis and making a billion dollar startup? Looking at compute or parameters, you might be able to say that no AI could achieve all that with less than X flops. I have no idea how you would find X. But at least those are clear unambiguous technical predictions.