I would actually prefer more handwaving on the logical probability distribution. Currently it’s a bit confusing to me, with what appears to be mixed notation for conditional probabilities and set comprehensions. And the specifics of how the probabilities are assigned are not necessary for later results. (For example, does it really matter that you’re checking for a contradiction in proofs less than some length L, rather than checking for a contradiction in the collection of proofs generated by a general proof-generating process with resources R?) You can just say ” assign some logical probability distribution that includes statements like ‘A produces c’.”
ED(X) := Σk 2-k PD(dk(X)).
Why not just use ED(X) = ΣX X PD(X)?
QS(H) := 2-L(H) (1 - e-t(H)/t)
This distribution has the troublesome property that if there is some short hypothesis for which t(H)=1, and t is fixed, then as you get more and more data this hypothesis never goes away. In fact, if it is short enough, it can continue to dominate the true hypothesis indefinitely. Now, one can think of ad-hoc ways to fix this, but what do you think a non-ad-hoc way would look like? I feel like the goal is to eliminate hypotheses that have been checked and found to not fit Y, while still retaining hypotheses that you are logically uncertain about.
I(Q0) := EQS(Emax(U(Y(H)) | “Q(Y(H)) = Q0” is true))
I’ll need some help unpacking this. I don’t know how this is supposed to get its dependence on Q0, rather than just being a function of U and Y. Is the idea that QS(H) is different if use different Q(Y)? Are you eliding some updating process that was supposed to be obvious to me? As you can tell I’m pretty confused.
I would actually prefer more handwaving on the logical probability distribution. Currently it’s a bit confusing to me, with what appears to be mixed notation for conditional probabilities and set comprehensions.
I’m somewhat confused as to nature of your confusion. Are you saying you don’t understand the definition? Or suggesting to generalize it?
Why not just use ED(X) = ΣX X PD(X)?
Because “PD(X0)” is problematic. What is “X0″? A dyadic fraction? For a long dyadic fraction X0, the statement “X=X0” is long and therefore assigned probability 1⁄2. This series doesn’t converge. Moreover, it seems just wrong to consider the exact equality of X to fixed dyadic fractions, especially considering that X can be provably not a dyadic fraction.
This distribution has the troublesome property that if there is some short hypothesis for which t(H)=1, and t is fixed, then as you get more and more data this hypothesis never goes away.
What makes you think so? As G gathers data, it will be able to eliminate hypotheses, including short ones.
I don’t know how this is supposed to get its dependence on Q0, rather than just being a function of U and Y.
It gets its dependence on Q0 from the condition “Q(Y(H)) = Q0” in the inner expectation value.
I’m somewhat confused as to nature of your confusion.
The main specific problem is the statement “PD(s) := P0(s | there are no contradictions of length ⇐ D).” This doesn’t make sense as conditionalization. Do you mean PD(s) = P0(s) if there are no contradictions of length<D? But even then, this doesn’t define a distribution—you need to further assume that PD(s)=1 if there’s a proof and PD(s)=0 if there’s a proof of not-s.
Why not just use ED(X) = ΣX X PD(X)?
This series doesn’t converge.
Good point. To fix this, you’d have to take advantage of information about mutual exclusivity when assigning logical probabilities. Restricting X to being less than 1 and looking at the digits is a good workaround to make it converge, though if you can’t prove what any of the digits will be (e.g. if you assign P=0.5 to very long sequences with arbitrary digits) you’ll just get that every digit is 50⁄50 1 and 0.
What makes you think so? As G gathers data, it will be able to eliminate hypotheses, including short ones.
Hmm, looks like I misunderstood. So are you assuming a certain bridging law in calculating t? That limits your ontology, though… But if you assume that you have the right bridging laws to test H, why do you need N?
It gets its dependence on Q0 from the condition “Q(Y(H)) = Q0” in the inner expectation value.
I can see that. And yet, I don’t know where that matters if you go step by step calculating the answer.
The main specific problem is the statement “PD(s) := P0(s | there are no contradictions of length ⇐ D).” This doesn’t make sense as conditionalization.
It does. The probability space consists of truth assignments. P0 is a probability distribution on truth assignments. “There are no contradictions of length ⇐ D” is a condition on truth assignments with positive P0-probability, so we can form the conditional probability distribution. You can think about it as follows: Toss a fair coin for every statement. If the resulting assignment contains contradictions of length ⇐ D, toss all coins again. With probability 1, the process will terminate after a finite number of tosses (since there is a positive probability for it to terminate on every step).
if you can’t prove what any of the digits will be (e.g. if you assign P=0.5 to very long sequences with arbitrary digits) you’ll just get that every digit is 50⁄50 1 and 0.
Not really. If you can decide the value of a digit at the given depth of analysis D, it will be 0 or 1. If you can’t it will have some probability to be 1, not necessarily 1⁄2. It only becomes 1⁄2 for digits at places > 2^D (roughly).
Hmm, looks like I misunderstood. So are you assuming a certain bridging law in calculating t?
No, t is just a “god given” constant. I don’t understand the reason for your concern. The factor 1 - e^(-t(H)/t) ⇐ 1 < infinity so it cannot lead to infinite confidence in a hypothesis.
I can see that. And yet, I don’t know where that matters if you go step by step calculating the answer.
I’m computing the utility under the logical counterfactual assumption that H produces Q0. Thus if Q0 is “designed to generate utility” in a sense, the resulting expected utility will be high, otherwise low. It’s just like in usual UDT.
You can think about it as follows: Toss a fair coin for every statement. If the resulting assignment contains contradictions of length ⇐ D, toss all coins again.
Thanks! I understand your usage now, that was a good explanation.
It only becomes 1⁄2 for digits at places > 2^D (roughly).
If you check consistency of statements with length less than D but allow proofs of infinite length, you’ll need infinite computational resources. That’s bad.
No, t is just a “god given” constant.
I meant “calculating t(H),” sorry. But anyhow, I think there are several possible examples of bad behavior.
I don’t understand the reason for your concern. The factor 1 - e^(-t(H)/t) ⇐ 1 < infinity so it cannot lead to infinite confidence in a hypothesis.
Suppose that N specifies that our agent is a turing machine. It describes a universe Y with in terms of some tapes that can be read or written to. N has some initial predictions about the contents of the tapes that may or may not be accurate. Each step of Y is encoded as a big number.
Now consider some other hypothesis H which is just Y=[1,2,3,4...]. We can offset the zero time so that H and N start with the same number, so that t(H)=1. And H is much, much simpler than N. How would your agent go about showing that H is wrong?
I’m computing the utility under the logical counterfactual assumption that H produces Q0. Thus if Q0 is “designed to generate utility” in a sense, the resulting expected utility will be high,
If you check consistency of statements with length less than D but allow proofs of infinite length, you’ll need infinite computational resources. That’s bad.
I don’t care about proofs. Only about “D-consistency”. The probability of s is the ratio of the number D-consistent truth assignments in which s is true to the total number of D-consistent truth assignments. When a short proof of s exists, all D-consistent truth assignments define s as true, so its probability is 1. When a short proof of “not s” exists, all D-consistent truth assignments define s a false, so its probability is 0. In all other cases the ratio is some number between 0 and 1, not necessarily 1⁄2.
Now consider some other hypothesis H which is just Y=[1,2,3,4...]. We can offset the zero time so that H and N start with the same number, so that t(H)=1. And H is much, much simpler than N. How would your agent go about showing that H is wrong?
For the agent to be functional, t has to be sufficiently large. For sufficiently large t, all hypotheses with small t(H) are suppressed, even the simplest ones. In fact, I suspect there is a certain critical t at which the agent gains the ability to accumulate knowledge over time.
I don’t care about proofs. Only about “D-consistency”.
Fair enough. But would you agree with the claim that a real-world agent is going to have to use a formulation that fits inside limits on the length of usable proofs? This circles back to my suggestion that the specifics aren’t that important here and a handwaved generic logical probability distribution would suffice :)
For the agent to be functional, t has to be sufficiently large. For sufficiently large t, all hypotheses with small t(H) are suppressed, even the simplest ones. In fact, I suspect there is a certain critical t at which the agent gains the ability to accumulate knowledge over time.
Hm. Upon further reflection, I think the problems are not with your distribution (which I had initially misinterpreted to be a posterior distribution, with t(H) a property of the data :P ), but with the neglect of bridging laws or different ways of representing the universe.
For example, if our starting ontology is classical mechanics and our universe at each time step is just a big number encoding the coordinates of the particles, quantum mechanics has a t(H)=0, because it’s such a different format. - it’s the coordinates of some point in Hilbert space, not phase space. Being able to rediscover quantum mechanics is important.
If you neglect bridging laws, then your hypotheses are either untestable, or only testable using some sort of default mapping (comparing the input channel of your agent to the input channel of Q(H), which needs to be found by interpreting Q(H) using a particular format). If our agent exists in the universe in a different format, then we need to specify some different way of finding its input channel.
Another problem from the neglect of bridging laws is that when the bridging laws themselves are highly complex, you want to penalize this. You can’t just have the universe be [1,2,3...] and then map those to the correct observations (using some big look-up table) and claim it’s a simple hypothesis.
But would you agree with the claim that a real-world agent is going to have to use a formulation that fits inside limits on the length of usable proofs?
I’m not defining an agent here, I’m defining a mathematical function which evaluates agents. It is uncomputable (as is the Legg-Hutter metric).
Upon further reflection, I think the problems are not with your distribution… but with the neglect of bridging laws or different ways of representing the universe.
N defines the ontology in which the utility function and the “intrinsic mind model” are defined. Y should be regarded as the projection of the universe on this ontology rather than the “objective universe” (whatever the latter means). Thus H implicitly includes both the model of the universe and the bridging laws. In particular, its complexity reflects the total complexity of both. For example, if N is classical and the universe is quantum mechanical, G will arrive at a hypothesis H which combines quantum mechanics with decoherence theory to produce classical macroscopic histories. This hypothesis will have large t(H) since quantum mechanics correctly reproduces the classical dynamics of M at the macroscopic level. This shouldn’t come as a surprise: we also perceive the world as classical. More precisely, there would be a dominant family of hypothesis differing in the results of “quantum coin tosses”. That is, this ontological projection is precisely the place where the probability interpretation of the wavefunction arises.
Nice!
I would actually prefer more handwaving on the logical probability distribution. Currently it’s a bit confusing to me, with what appears to be mixed notation for conditional probabilities and set comprehensions. And the specifics of how the probabilities are assigned are not necessary for later results. (For example, does it really matter that you’re checking for a contradiction in proofs less than some length L, rather than checking for a contradiction in the collection of proofs generated by a general proof-generating process with resources R?) You can just say ” assign some logical probability distribution that includes statements like ‘A produces c’.”
Why not just use ED(X) = ΣX X PD(X)?
This distribution has the troublesome property that if there is some short hypothesis for which t(H)=1, and t is fixed, then as you get more and more data this hypothesis never goes away. In fact, if it is short enough, it can continue to dominate the true hypothesis indefinitely. Now, one can think of ad-hoc ways to fix this, but what do you think a non-ad-hoc way would look like? I feel like the goal is to eliminate hypotheses that have been checked and found to not fit Y, while still retaining hypotheses that you are logically uncertain about.
I’ll need some help unpacking this. I don’t know how this is supposed to get its dependence on Q0, rather than just being a function of U and Y. Is the idea that QS(H) is different if use different Q(Y)? Are you eliding some updating process that was supposed to be obvious to me? As you can tell I’m pretty confused.
I’m somewhat confused as to nature of your confusion. Are you saying you don’t understand the definition? Or suggesting to generalize it?
Because “PD(X0)” is problematic. What is “X0″? A dyadic fraction? For a long dyadic fraction X0, the statement “X=X0” is long and therefore assigned probability 1⁄2. This series doesn’t converge. Moreover, it seems just wrong to consider the exact equality of X to fixed dyadic fractions, especially considering that X can be provably not a dyadic fraction.
What makes you think so? As G gathers data, it will be able to eliminate hypotheses, including short ones.
It gets its dependence on Q0 from the condition “Q(Y(H)) = Q0” in the inner expectation value.
I’ll gladly help as I can.
The main specific problem is the statement “PD(s) := P0(s | there are no contradictions of length ⇐ D).” This doesn’t make sense as conditionalization. Do you mean PD(s) = P0(s) if there are no contradictions of length<D? But even then, this doesn’t define a distribution—you need to further assume that PD(s)=1 if there’s a proof and PD(s)=0 if there’s a proof of not-s.
Good point. To fix this, you’d have to take advantage of information about mutual exclusivity when assigning logical probabilities. Restricting X to being less than 1 and looking at the digits is a good workaround to make it converge, though if you can’t prove what any of the digits will be (e.g. if you assign P=0.5 to very long sequences with arbitrary digits) you’ll just get that every digit is 50⁄50 1 and 0.
Hmm, looks like I misunderstood. So are you assuming a certain bridging law in calculating t? That limits your ontology, though… But if you assume that you have the right bridging laws to test H, why do you need N?
I can see that. And yet, I don’t know where that matters if you go step by step calculating the answer.
It does. The probability space consists of truth assignments. P0 is a probability distribution on truth assignments. “There are no contradictions of length ⇐ D” is a condition on truth assignments with positive P0-probability, so we can form the conditional probability distribution. You can think about it as follows: Toss a fair coin for every statement. If the resulting assignment contains contradictions of length ⇐ D, toss all coins again. With probability 1, the process will terminate after a finite number of tosses (since there is a positive probability for it to terminate on every step).
Not really. If you can decide the value of a digit at the given depth of analysis D, it will be 0 or 1. If you can’t it will have some probability to be 1, not necessarily 1⁄2. It only becomes 1⁄2 for digits at places > 2^D (roughly).
No, t is just a “god given” constant. I don’t understand the reason for your concern. The factor 1 - e^(-t(H)/t) ⇐ 1 < infinity so it cannot lead to infinite confidence in a hypothesis.
I’m computing the utility under the logical counterfactual assumption that H produces Q0. Thus if Q0 is “designed to generate utility” in a sense, the resulting expected utility will be high, otherwise low. It’s just like in usual UDT.
Thanks! I understand your usage now, that was a good explanation.
If you check consistency of statements with length less than D but allow proofs of infinite length, you’ll need infinite computational resources. That’s bad.
I meant “calculating t(H),” sorry. But anyhow, I think there are several possible examples of bad behavior.
Suppose that N specifies that our agent is a turing machine. It describes a universe Y with in terms of some tapes that can be read or written to. N has some initial predictions about the contents of the tapes that may or may not be accurate. Each step of Y is encoded as a big number.
Now consider some other hypothesis H which is just Y=[1,2,3,4...]. We can offset the zero time so that H and N start with the same number, so that t(H)=1. And H is much, much simpler than N. How would your agent go about showing that H is wrong?
Yay, I’m less confused now.
I don’t care about proofs. Only about “D-consistency”. The probability of s is the ratio of the number D-consistent truth assignments in which s is true to the total number of D-consistent truth assignments. When a short proof of s exists, all D-consistent truth assignments define s as true, so its probability is 1. When a short proof of “not s” exists, all D-consistent truth assignments define s a false, so its probability is 0. In all other cases the ratio is some number between 0 and 1, not necessarily 1⁄2.
For the agent to be functional, t has to be sufficiently large. For sufficiently large t, all hypotheses with small t(H) are suppressed, even the simplest ones. In fact, I suspect there is a certain critical t at which the agent gains the ability to accumulate knowledge over time.
Fair enough. But would you agree with the claim that a real-world agent is going to have to use a formulation that fits inside limits on the length of usable proofs? This circles back to my suggestion that the specifics aren’t that important here and a handwaved generic logical probability distribution would suffice :)
Hm. Upon further reflection, I think the problems are not with your distribution (which I had initially misinterpreted to be a posterior distribution, with t(H) a property of the data :P ), but with the neglect of bridging laws or different ways of representing the universe.
For example, if our starting ontology is classical mechanics and our universe at each time step is just a big number encoding the coordinates of the particles, quantum mechanics has a t(H)=0, because it’s such a different format. - it’s the coordinates of some point in Hilbert space, not phase space. Being able to rediscover quantum mechanics is important.
If you neglect bridging laws, then your hypotheses are either untestable, or only testable using some sort of default mapping (comparing the input channel of your agent to the input channel of Q(H), which needs to be found by interpreting Q(H) using a particular format). If our agent exists in the universe in a different format, then we need to specify some different way of finding its input channel.
Another problem from the neglect of bridging laws is that when the bridging laws themselves are highly complex, you want to penalize this. You can’t just have the universe be [1,2,3...] and then map those to the correct observations (using some big look-up table) and claim it’s a simple hypothesis.
I’m not defining an agent here, I’m defining a mathematical function which evaluates agents. It is uncomputable (as is the Legg-Hutter metric).
N defines the ontology in which the utility function and the “intrinsic mind model” are defined. Y should be regarded as the projection of the universe on this ontology rather than the “objective universe” (whatever the latter means). Thus H implicitly includes both the model of the universe and the bridging laws. In particular, its complexity reflects the total complexity of both. For example, if N is classical and the universe is quantum mechanical, G will arrive at a hypothesis H which combines quantum mechanics with decoherence theory to produce classical macroscopic histories. This hypothesis will have large t(H) since quantum mechanics correctly reproduces the classical dynamics of M at the macroscopic level. This shouldn’t come as a surprise: we also perceive the world as classical. More precisely, there would be a dominant family of hypothesis differing in the results of “quantum coin tosses”. That is, this ontological projection is precisely the place where the probability interpretation of the wavefunction arises.