Check out the link to efficient optimization at the top of the post, he never said anything about running statistics when measuring intelligence.
What we want is something that will enable a program to judge the quality of another so it will always be better when it picks a new program to rewrite to, so it can be take over the galaxy. Feel free to pick holes in this notion if you want, it is not mine.
I might rewrite the post to have the notion of strict ordering. A system is better than another if will do no worse under all physically plausible environments and is better in some. Then Mu would allow you to setup two programs that are differently ordered and ask you to bet on their ordering.
You really don’t want to do statistics on the first seed AI if it is possible.
Then you’re arguing that, if your notion of “physically plausible environments” includes a certain class of adversely optimized situations, worst-case analysis won’t work because all worst cases are equally bad.
By “how well a system would do in physically plausible environments”, do you mean on average, worst case or something else?
Check out the link to efficient optimization at the top of the post, he never said anything about running statistics when measuring intelligence.
What we want is something that will enable a program to judge the quality of another so it will always be better when it picks a new program to rewrite to, so it can be take over the galaxy. Feel free to pick holes in this notion if you want, it is not mine.
I might rewrite the post to have the notion of strict ordering. A system is better than another if will do no worse under all physically plausible environments and is better in some. Then Mu would allow you to setup two programs that are differently ordered and ask you to bet on their ordering.
You really don’t want to do statistics on the first seed AI if it is possible.
Then you’re arguing that, if your notion of “physically plausible environments” includes a certain class of adversely optimized situations, worst-case analysis won’t work because all worst cases are equally bad.
They could all be vaporised by a near super nova or something similar before they have a chance to do anything, yup.