If the AGI is using machinery that would allow it to simulate any world-model, it will be way slower than the Turing machine built for that algorithm.
I don’t buy it. All your programs are already running on UTM M.
Just consider a program that gives the aliens the ability to write arbitrary functions in M and then pass control to them. That program is barely any bigger (all you have to do is insert one use after free in physics :) ), and guarantees the aliens have zero slowdown.
For the literal simplest version of this, your program is M(Alien(), randomness), which is going to run just as fast as M(physics, randomness) for the intended physics, and probably much faster (if the aliens can think of any clever tricks to run faster without compromising much accuracy). The only reason you wouldn’t get this is if Alien is expensive. That probably rules out crazy alien civilizations, but I’m with Wei Dai that it probably doesn’t rule out simpler scientists.
Just consider a program that gives the aliens the ability to write arbitrary functions in M and then pass control to them.
That’s what I was thinking too, but Michael made me realize this isn’t possible, at least for some M. Suppose M is the C programming language, but in C there is no way to say “interpret this string as a C program and run it as fast as a native C program”. Am I missing something at this point?
all you have to do is insert one use after free in physics
That’s what I was thinking too, but Michael made me realize this isn’t possible, at least for some M. Suppose M is the C programming language, but in C there is no way to say “interpret this string as a C program and run it as fast as a native C program”. Am I missing something at this point?
I agree this is only going to be possible for some universal Turing machines. Though if you are using a Turing machine to define a speed prior, this does seem like a desirable property.
I don’t understand this sentence.
If physics is implemented in C, there are many possible bugs that would allow the attacker to execute arbitrary C code with no slowdown.
I agree this is only going to be possible for some universal Turing machines. Though if you are using a Turing machine to define a speed prior, this does seem like a desirable property.
Why is it a desirable property? I’m not seeing why it would be bad to choose a UTM that doesn’t have this property to define the speed prior for BoMAI, if that helps with safety. Please explain more?
I just mean: “universality” in the sense of a UTM isn’t a sufficient property when defining the speed prior, the analogous property of the UTM is something more like: “You can run an arbitrary Turing machine without too much slowdown.” Of course that’s not possible, but it seems like you still want to be as close to that as possible (for the same reasons that you wanted universality at all).
I agree that it would be fine to sacrifice this property if it was helpful for safety.
Each world-model is a Turing machine, whose prior relates to the Kolmogorov complexity (on some universal Turing machine) of the description of Turing machine—all the transition rules, and whatnot. Usually, this would be isomorphic (within a constant), but since we’re considering speed, programs actually aren’t simulated on a UTM.
I don’t buy it. All your programs are already running on UTM M.
Just consider a program that gives the aliens the ability to write arbitrary functions in M and then pass control to them. That program is barely any bigger (all you have to do is insert one use after free in physics :) ), and guarantees the aliens have zero slowdown.
For the literal simplest version of this, your program is M(Alien(), randomness), which is going to run just as fast as M(physics, randomness) for the intended physics, and probably much faster (if the aliens can think of any clever tricks to run faster without compromising much accuracy). The only reason you wouldn’t get this is if Alien is expensive. That probably rules out crazy alien civilizations, but I’m with Wei Dai that it probably doesn’t rule out simpler scientists.
That’s what I was thinking too, but Michael made me realize this isn’t possible, at least for some M. Suppose M is the C programming language, but in C there is no way to say “interpret this string as a C program and run it as fast as a native C program”. Am I missing something at this point?
I don’t understand this sentence.
I agree this is only going to be possible for some universal Turing machines. Though if you are using a Turing machine to define a speed prior, this does seem like a desirable property.
If physics is implemented in C, there are many possible bugs that would allow the attacker to execute arbitrary C code with no slowdown.
Why is it a desirable property? I’m not seeing why it would be bad to choose a UTM that doesn’t have this property to define the speed prior for BoMAI, if that helps with safety. Please explain more?
I just mean: “universality” in the sense of a UTM isn’t a sufficient property when defining the speed prior, the analogous property of the UTM is something more like: “You can run an arbitrary Turing machine without too much slowdown.” Of course that’s not possible, but it seems like you still want to be as close to that as possible (for the same reasons that you wanted universality at all).
I agree that it would be fine to sacrifice this property if it was helpful for safety.
Each world-model is a Turing machine, whose prior relates to the Kolmogorov complexity (on some universal Turing machine) of the description of Turing machine—all the transition rules, and whatnot. Usually, this would be isomorphic (within a constant), but since we’re considering speed, programs actually aren’t simulated on a UTM.