Slightly tangential question about source code swapping:
If A’s source code depends on what it reads from B, and B on what it reads from A…
Is there any chance of a Halting problem?
No. A’s action depends on the soucecode of B and vice versa. As the sourcecode does not depend upon the sourcecode, nor the action on the action, you aren’t into recursion
Let’s say I write a program A that compiles and runs B with A’s source code as input. If B’s output is “cooperate”, A cooperates. If B’s output is “defect”, A defects.
Now I pit A against itself. Oops, infinite recursion. This isn’t exactly the same as the halting problem, but it rhymes.
Are you talking about scenario 1 or 2? In scenario 1, A doesn’t run B, it only checks that the beginning of B’s source code matches a certain string (which happens to coincide with the beginning of A’s source code). In scenario 2, note my note about the random number generator.
I’m talking about scenario 1. I am not talking about your version of A and B. I assumed A and B generally referred to the two programs in the competition. Anyway, I gave an example of a program that would behave optimally. Unfortunately, it will go into infinite recursion against any program using a similar tactic.
Basically, If you know a program’s source code and you know what input it will receive, you can perfectly predict the output.
I don’t want to be rude, but I can’t understand why anyone is arguing otherwise. Scenario 1 is very similar to the halting problem.
I wouldn’t call a program that fails to cooperate with itself “optimal”. My program is more optimal than yours, because it cooperates with itself in finite time. :-)
But this is an interesting direction of inquiry too. Is there a non-contradictory way to add something like magical predictor oracles to scenario 1? Would the resulting problem be mathematically interesting? Eliezer seems to think yes, unless I’ve misunderstood his position… but he refuses to disclose the math.
In general, in the scenarios like 2, you should give info other than black-box access to running. There should be a way to abstractly model the other player. Following something like Solomonoff induction is the last resort.
Slightly tangential question about source code swapping: If A’s source code depends on what it reads from B, and B on what it reads from A… Is there any chance of a Halting problem?
No. A’s action depends on the soucecode of B and vice versa. As the sourcecode does not depend upon the sourcecode, nor the action on the action, you aren’t into recursion
Let’s say I write a program A that compiles and runs B with A’s source code as input. If B’s output is “cooperate”, A cooperates. If B’s output is “defect”, A defects.
Now I pit A against itself. Oops, infinite recursion. This isn’t exactly the same as the halting problem, but it rhymes.
Are you talking about scenario 1 or 2? In scenario 1, A doesn’t run B, it only checks that the beginning of B’s source code matches a certain string (which happens to coincide with the beginning of A’s source code). In scenario 2, note my note about the random number generator.
I’m talking about scenario 1. I am not talking about your version of A and B. I assumed A and B generally referred to the two programs in the competition. Anyway, I gave an example of a program that would behave optimally. Unfortunately, it will go into infinite recursion against any program using a similar tactic.
Basically, If you know a program’s source code and you know what input it will receive, you can perfectly predict the output.
I don’t want to be rude, but I can’t understand why anyone is arguing otherwise. Scenario 1 is very similar to the halting problem.
I wouldn’t call a program that fails to cooperate with itself “optimal”. My program is more optimal than yours, because it cooperates with itself in finite time. :-)
But this is an interesting direction of inquiry too. Is there a non-contradictory way to add something like magical predictor oracles to scenario 1? Would the resulting problem be mathematically interesting? Eliezer seems to think yes, unless I’ve misunderstood his position… but he refuses to disclose the math.
In general, in the scenarios like 2, you should give info other than black-box access to running. There should be a way to abstractly model the other player. Following something like Solomonoff induction is the last resort.