For CDT, it is not useful to try simulating every single piece of code within U(), since there is no Correlator to use the results. So the simplest version of the CDT agent would be:
Let L = available computation resources / number of functions in U(). Evaluate U(), spending at most L steps in each function. If a function takes too long to evaluate, assume its result is a new variable x_i and continue to evaluate its parent. In the end, the final result should be: U() = some function of variables x_i, one of which is the variable corresponding to Self(). Return the argmax of that function (assuming some distribution over non-self variables, if necessary).
I don’t understand. In the symmetric PD the world program makes two identical calls to the agent. If the agent chooses defection, does that mean it randomly designates one of these calls as “myself” and the other as “not myself”?
The cases where the two calls are to the same Self() function correspond to “anthropic uncertainty” in the Newcomb’s problem and “playing against mirror image” in PD. In these cases even CDT knows that it directly influences both parts, and so it one-boxes/cooperates. For properly modeling the symmetric PD, the second agent must be a function with a different name but with the same source code.
So a CDT agent assumes some probability distribution over the outputs of all computations except itself, then chooses a return value by maximizing expected utility under that distribution. In the PD and Newcomb’s problem, assuming any reasonable probability distribution will lead to defecting or two-boxing. That’s a nice way of looking at it, thanks.
So a CDT agent assumes some probability distribution over the outputs of all computations except itself
Yes, all computations that it has no resources to directly simulate. Which necessarily include any copies of itself, no matter how much resources it has.
What do you think of the other idea—that all ways of recognizing correlations can be reduced to series of evaluations and source code comparisons? I don’t immediately see any other way to prove that two source codes return the same thing… must check a textbook on lambda calculus.
No, that idea looks wrong, unless I misunderstand it. Imagine you have two functions f() and g() that you’re too weak to evaluate. You need to be able to prove that f()+g()==g()+f(), but I don’t see any way to do that using only evaluations and source code comparisons. The general way is to use a theorem prover that has axioms like the commutativity of addition.
Don’t you need a theorem prover to go from commutativity of addition to f()+g()==g()+f()? I thought you were supposed to use only evaluations and source code comparisons on the code f()+g()==g()+f() directly...
Commutativity of addition is a correlation P1=P2, where P1(x,y)=x+y, and P2(x,y)=y+x. So, f()+g() = [using this correlation] = g()+f().
But you’re right. In order to use this, I have to be able to calculate evaluations and comparisons on arbitrary pieces of code, not just the ones existing in U()...
If you have a formalization of CDT in the constant programs framework, could you spell it out in more detail?
For CDT, it is not useful to try simulating every single piece of code within U(), since there is no Correlator to use the results. So the simplest version of the CDT agent would be:
Let L = available computation resources / number of functions in U().
Evaluate U(), spending at most L steps in each function. If a function takes too long to evaluate, assume its result is a new variable x_i and continue to evaluate its parent.
In the end, the final result should be: U() = some function of variables x_i, one of which is the variable corresponding to Self().
Return the argmax of that function (assuming some distribution over non-self variables, if necessary).
I don’t understand. In the symmetric PD the world program makes two identical calls to the agent. If the agent chooses defection, does that mean it randomly designates one of these calls as “myself” and the other as “not myself”?
The cases where the two calls are to the same Self() function correspond to “anthropic uncertainty” in the Newcomb’s problem and “playing against mirror image” in PD. In these cases even CDT knows that it directly influences both parts, and so it one-boxes/cooperates. For properly modeling the symmetric PD, the second agent must be a function with a different name but with the same source code.
Oh. You’re right.
So a CDT agent assumes some probability distribution over the outputs of all computations except itself, then chooses a return value by maximizing expected utility under that distribution. In the PD and Newcomb’s problem, assuming any reasonable probability distribution will lead to defecting or two-boxing. That’s a nice way of looking at it, thanks.
Yes, all computations that it has no resources to directly simulate. Which necessarily include any copies of itself, no matter how much resources it has.
What do you think of the other idea—that all ways of recognizing correlations can be reduced to series of evaluations and source code comparisons? I don’t immediately see any other way to prove that two source codes return the same thing… must check a textbook on lambda calculus.
No, that idea looks wrong, unless I misunderstand it. Imagine you have two functions f() and g() that you’re too weak to evaluate. You need to be able to prove that f()+g()==g()+f(), but I don’t see any way to do that using only evaluations and source code comparisons. The general way is to use a theorem prover that has axioms like the commutativity of addition.
I think I can prove commutativity of addition using just evaluations and source code comparisons:
def P(m,n): P(m, 0) = m ; P(m, n+1) = P(m, n) + 1
Lemma: P(m+1, n) = P(m, n+1)
Proof:
Let A(m,n) = P(m+1, n),
Let B(m,n) = P(m, n+1)
Then:
A(m, 0) = P(m+1, 0) = m+1
A(m, n+1) = P(m+1, n+1) = P(m+1, n) + 1 = A(m, n) + 1
B(m, 0) = P(m, 0+1) = P(m) + 1 = m+1
B(m, n+1) = P(m, n+1+1) = P(m, n+1) + 1 = B(m, n) + 1
The source code for A and B are equivalent ⇒ A=B. QED
Theorem: P(m, n) = P(n, m)
Proof:
P(0, 0) = 0
P(m+1, 0) = P(m, 0) + 1
P(0, n+1) = P(0, n) + 1
P(m+1, n+1) = P(m+1, n) + 1 = P(m, n+1) + 1 = P(m, n) + 1 + 1
The code of P is symmetrical now, so P(m,n)=P(n,m). QED
Don’t you need a theorem prover to go from commutativity of addition to f()+g()==g()+f()? I thought you were supposed to use only evaluations and source code comparisons on the code f()+g()==g()+f() directly...
Commutativity of addition is a correlation P1=P2, where P1(x,y)=x+y, and P2(x,y)=y+x.
So, f()+g() = [using this correlation] = g()+f().
But you’re right. In order to use this, I have to be able to calculate evaluations and comparisons on arbitrary pieces of code, not just the ones existing in U()...