Rice’s theorem says nothing about the impossibility to show properties of programs in special cases. It only shows that there are always some programs which will thwart your decision procedure. But your possible AI does not need to be one of them.
So where do we go? We go kicking Mu out of the picture:
For the most interesting aspect of an intelligent system is whether it is efficient in the very world it will occupy. So a reasonable test of it’s intelligence is to run it in a detailed simulation of said world. Of course, if your simulated world and reality differ (for example in the existence of Mu), the measure is pretty much worthless.
However, decent simulations of our world can be built.
Also, the solution to your puzzle is to copy the world (including Mu) into a new Matrix, testing whether the program reaches it’s goal, and using that as your efficiency measure. That way Mu cannot test the program’s efficiency either, and you get a fair chance.
Rice’s theorem says nothing about the impossibility to show properties of programs in special cases. It only shows that there are always some programs which will thwart your decision procedure. But your possible AI does not need to be one of them.
So where do we go? We go kicking Mu out of the picture:
For the most interesting aspect of an intelligent system is whether it is efficient in the very world it will occupy. So a reasonable test of it’s intelligence is to run it in a detailed simulation of said world. Of course, if your simulated world and reality differ (for example in the existence of Mu), the measure is pretty much worthless.
However, decent simulations of our world can be built.
Also, the solution to your puzzle is to copy the world (including Mu) into a new Matrix, testing whether the program reaches it’s goal, and using that as your efficiency measure. That way Mu cannot test the program’s efficiency either, and you get a fair chance.