That’s unclear. Does the mugger show you the set of deals before you accept the offer or later? If he shows you later, he can trivially win by presenting you the set { ‘while (true) {printf(“I Go Chop Your Dollar\n”);}’ }
I think that even if the mugger doesn’t show you the set of deals before you accept there is still a non-zero probability that some of the deals have BB(N) utility unless the agent has p(mugger_is_perfect) = 1. I agree that I should have specified that in the description.
d(N) either halts or it doesn’t. There is nothing probabilistic about it. Are you talking about the prior probability about its termination before you see d(N), or after you see it (since you can’t compute its termination in the general case)?
The idea was to talk about the agent’s total estimation of the probability that a particular deal yields BB(N) utils. Before seeing it the agent would only have a prior probability based perhaps on Solomonoff Induction, and after seeing it the agent could just run it for a while to make sure it didn’t halt quickly. Either way, the idea was that any probability estimate would be dwarfed by BB(N).
p(UTM(d(N)) = BB(N)) BB(N) > 0 is trivially true because the product of two positive numbers is always positive. But this tells you nothing about the magnitude of the value. In particular, it doesn’t follow that
p(UTM(d(N)) = BB(N)) BB(N) >= U($5)
I think that even if the mugger doesn’t show you the set of deals before you accept there is still a non-zero probability that some of the deals have BB(N) utility unless the agent has p(mugger_is_perfect) = 1. I agree that I should have specified that in the description.
The idea was to talk about the agent’s total estimation of the probability that a particular deal yields BB(N) utils. Before seeing it the agent would only have a prior probability based perhaps on Solomonoff Induction, and after seeing it the agent could just run it for a while to make sure it didn’t halt quickly. Either way, the idea was that any probability estimate would be dwarfed by BB(N).
You are correct.