Isn’t that like saying you got Dutch-booked for assigning 1⁄2 as the probability of heads (because a clairvoyant will be able to foresee that the coin is actually falling tails)?
The relevant point is that, in real life, computation requirements can keep you from calculating the exact Bayesian probability, which will lead to dutch-booking if an agent with enough computing power has a good model of the approximation you’re using.
Collatz conjecture is true in every universe, or false in every universe.
You can slice it into a set of trivial statements which are trivially true or trivially false, like “Collatz conjecture is true for N=531” etc., connected by trivially true or trivially false logical statements.
There’s no way to meaningfully get any probability but 0 or 1 out of this, other than by claiming that some basic mathematical law is uncertain (and if you believe that, you are more Dutch bookable than entire Netherlands). I might not know how to Dutch book you yet, but logic dictates such a way exists.
Except thanks to Incompleteness Theorem, you have no way to find a definite answer to every such statement. No matter which strategy you choose, and how much time you have, you’ll either be inconsistent (Dutch bookable), or incomplete (not able to answer 0 or 1 - and as no other answer is valid, any answer you give makes you Dutch bookable).
Do you assign probability 1 to the proposition that 182547553 is prime? Right now, without doing an experiment on a calculator? (well, computer maybe. For most calculators testing this proposition would be somewhat tedious)
If yes, would you willing to pay me $10 if you ever found out it was not prime?
Conversely
Do you assign probability 0 to the proposition that 182547553 is prime? Right now, without doing an experiment on a calculator?
If yes, would you willing to pay me $10 if you ever found out it was prime?
EDIT: Actually, I suppose this counts as “doing it again”, even though I’m not Peter de Blanc. I think that makes me a bad bad person.
I suggest you look up the concept of “subjective Bayesian”. Probabilities are states of knowledge. If you don’t know an answer, it’s uncertain. If someone who doesn’t know anything you don’t can look over your odds and construct a knowably losing bet anyway, or construct a winning bet that you refuse, then you are Dutch-bookable.
Also, considering that you have apparently been reading this site for years and you still have not grasped the concept of subjective uncertainty, and you are still working with a frequentist notion of probability, nor yet have you even grasped the difference, I would suggest to you in all seriousness that you seek enlightenment elsewhere.
(Sorry, people, there’s got to be some point at which I can express that. Downvote if you must, I suppose, if you think such a concept is unspeakable or unallowed, but it’s not a judgment based on only one case of incomprehension, of course.)
I don’t know the background conflict. But at least one of taw’s points is correct. Any prior P, of any agent, has at least one of the following three properties:
It is not defined on all X—i.e. P is agnostic about some things
It has P(X) < P(X and Y) for at least one pair X and Y—i.e. P sometimes falls for the conjunction fallacy
It has P(X) = 1 for all mathematically provable statements X—i.e. P is an oracle.
You aren’t excused from having to pick one by rejecting frequentist theory.
To make use of a theory like probability one doesn’t have to have completely secure foundations. But it’s responsible to know what the foundational issues are. If you make a particularly dicey or weird application of probability theory, e.g. game theory with superintelligences, you should be prepared to explain (especially to yourself!) why you don’t expect those foundational issues to interfere with your argument.
About taw’s point in particular, I guess it’s possible that von Neumann gave a completely satisfactory solution when he was a teenager, or whatever, and that I’m just showcasing my ignorance. (I would dearly like to hear about this solution!) Otherwise your comment reads like you’re shooting the messenger.
Logical uncertainty is an open problem, of course (I attended a workshop on it once, and was surprised at how little progress has been made).
But so far as Dutch-booking goes, the obvious way out is 2 with the caveat that the probability distribution never has P(X) < (PX&Y) at the same time, i.e., you ask it P(X), it gives you an answer, you ask it P(X&Y), it gives you a higher answer, you ask it P(X) again and it now gives you an even higher answer from having updated its logical uncertainty upon seeing the new thought Y.
It is also clear from the above that taw does not comprehend the notion of subjective uncertainty, hence the notion of logical uncertainty.
How, exactly, to deal with logical uncertainty is an unsolved problem, no?
It’s not clear why it’s more of a problem for Bayesian than anyone else.
The post you linked to is a response to Eliezer arguing with Hanson, with Eliezer taking a pro-his-own-contrarianism stance. Of course he’s aware of that contrarianism.
How, exactly, to deal with logical uncertainty is an unsolved problem, no?
Your choice is either accepting that you will be sometimes inconsistent,
or accepting that you will sometimes answer “I don’t know” without providing a specific number, or both.
There’s nothing wrong with “I don’t know”.
It’s not clear why it’s more of a problem for Bayesian than anyone else.
For Perfect Bayesian or for Subjective Bayesian?
Subjective Bayesian does believe many statements of kind P(simple math step) = 1, P(X|conjunction of simple math steps) = 1, and yet P(X) < 1.
it does not believe math statements with probability 1 or 0 until it investigates them. As soon as it investigates whether (X|conjunction of simple math steps) is true and determines the answer, it sets P(X)=1.
The problem with “I don’t know” is that sometimes you have to make decisions. How do you propose to make decisions if you don’t know some relevant mathematical fact X?
For example, if you’re considering some kind of computer security system that is intended to last a long time, you really need an estimate for how likely it is that P=NP.
The problem with “I don’t know” is that sometimes you have to make decisions. How do you propose to make decisions if you don’t know some relevant mathematical fact X?
Then you need to fully accept that you will be inconsistent sometimes. And compartmentalize your belief system accordingly, or otherwise find a way to deal with these inconsistencies.
Well, they can wriggle out of this by denying P(simple math step) = 1
Doesn’t this imply you’d be willing to accept P(2+2=5) on good enough odds?
This might be pragmatically a reasonable thing to do, but if you accept that all math might be broken, you’ve already given up any hope of consistency.
If physics is deterministic then conditional on the state of the world at the time you make the bet, the probability of heads is either 0 or 1. The only disanalogy with your example is that you may not already have sufficient information to determine how the coin will land (which isn’t even a disanalogy if we assume that the person doesn’t know what the Collatz conjecture says). But suppose you did have that information—there would be vastly more of it than you could process in the time available, so it wouldn’t affect your probability assignments. (Note: The case where the Collatz conjecture turns out to be true but unprovable is analogous to the case where the laws of physics are deterministic but ‘uncomputable’ in some sense.)
Anyway, the real reason why I want to resist your line of argument here is due to Chaitin’s number “omega”, the “halting probability”. One can prove that the bits in the binary expansion of omega are algorithmically incompressible. More precisely: In order to deduce n bits’ worth of information about omega you need at least n—k bits’ worth of axioms, for some constant k. Hence, if you look sufficiently far along the binary expansion of omega, you find yourself looking at an infinite string of “random mathematical facts”. One “ought” to treat these numbers as having probability 1⁄2 of being 1 and 1⁄2 of being 0 (if playing games against opponents who lack oracle powers).
Deterministic world and “too much information to process” are uninteresting. All that simply means that due to practical constraints, sometimes the only reasonably thing is to assign no probability. As if we didn’t know that already. But probabilities still might be assignable in theory.
Except uncomputability means it won’t work even in theory. You are always Dutch bookable.
Anyway, the real reason why I want to resist your line of argument here is due to Chaitin’s number “omega”, the “halting probability”. One can prove that the bits in the binary expansion of omega are algorithmically incompressible.
Chaitin’s number is not a mathematical entity—it’s creation of pure metaphysics.
The claim that kth bit of Chaitin’s number is 0 just doesn’t mean anything once k becomes big enough to include a procedure to compute Chaitin’s number.
Deterministic world and “too much information to process” are uninteresting. All that simply means that due to practical constraints, sometimes the only reasonably thing is to assign no probability.
Better: Sometimes the only reasonable thing is to assign a probability that’s strictly speaking “wrong”, but adequate if you’re only facing opponents who are (approximately) as hampered as you in terms of how much they know and much they can feasible compute. (E.g. Like humans playing poker, where the cards are only pseudo-random.)
If you want to say this is uninteresting, fine. I’m not trying to argue that it’s interesting.
Except uncomputability means it won’t work even in theory. You are always Dutch bookable.
Sorry, you’ve lost me.
Chaitin’s number is not a mathematical entity—it’s creation of pure metaphysics.
Chaitin’s number is awfully tame by the standards of descriptive set theory. So what you’re really saying here is that you personally regard a whole branch of mathematics as “pure metaphysics”. Maybe a few philosophers of mathematics agree with you—I suspect most do not—but actual mathematicians will carry on studying mathematics regardless.
The claim that kth bit of Chaitin’s number is 0 just doesn’t mean anything once k becomes big enough to include a procedure to compute Chaitin’s number.
I’m not sure what you’re trying to say here but what you’ve actually written is false. Why do you think Chaitin’s number isn’t well defined?
For what it’s worth, I think you made a very interesting contribution to this thread, and I find it somewhat baffling that EY responded in the way he did (though perhaps there’s some ‘history’ here that I’m not aware of) and equally baffling that this has apparently caused others to downvote you.
Do you understand how any definite answer other than 0 and 1 means you just got successfully Dutch booked?
No. I don’t understand that. Could you sketch the argument?
Incidentally, this may be playing with words, but the usual way of expressing the Dutch-book-victimhood of the hapless non-Bayesian agent is to say that a Dutch book can be constructed against him—not merely that a Dutch book exists. Successful gamblers engage in constructive, computable mathematics.
There is no hope for LessWrong as long as people keep conflating Perfect Bayesian and Subjective Bayesian.
Let’s take Subjective Bayesian first. The problem is—Subjective Bayesian breaks basic laws of probability as a routine matter.
Take the simplest possible law of probability P(X) >= P(X and Y).
Now let’s X be any mathematical theorem which you’re not sure about. 1 > P(X) > 0.
Let Y be some kind of “the following proof of X is correct”.
Verifying proofs is usually very simple, so very once you’re asked about P(X and Y), you can confidently reply P(X and Y) = 1. Y is not a new information about the world. It is usually conjunction of trivial statements which you already assigned probability 1.
That is—there’s infinite number of statements for which Subjective Bayesian will reply P(X) < P(X and Y).
For Subjective Bayesian X doesn’t even have to involve any infinities, just ask a simple question about cryptography which is pretty much guaranteed to be unsolvable before heat death of the universe, and you’re done.
At this point people far too often try to switch Perfect Bayesian for Subjective Bayesian.
And this is true, Perfect Bayesian wouldn’t make this particular mistake, and all probabilities of mathematical theorems he’d give would be 0 or 1, no exceptions. The problem is—Perfect Bayesians are not possible due to uncomputability.
If your version of Perfect Bayesian is computable, straightforward application of Rice Theorem shows he won’t be able to answer every question consistently.
If you claim some super-computable oracle version Perfect Bayesian—well, first that’s already metaphysics not mathematics, but in the end, this way of working around uncomputability does not work.
At any mention of uncomputability people far too often try to switch Subjective Bayesian for Perfect Bayesian (see Eliezer’s comment).
Excellent—now you’ve explained what you mean by “Perfect Bayesian” it all makes sense! (Though I can’t help thinking it would have saved time if you’d said this earlier.)
Still, I’m not keen on this redefinition of the word ‘metaphysics’, as though your philosophy of mathematics were ‘obviously correct’ ‘received wisdom’, when actually it’s highly contentious.
Anyway, I think this is a successful attack on a kind of “Bayesian absolutism” which claims that beings who (explicitly or implicitly) assign consistent probabilities to all expressible events, and update their beliefs in the Bayesian manner, can actually exist. That may be a straw man, though.
Imagine taking the proofs of all provable propositions that can be expressed in less than N characters (for some very large number N), writing them as conjunctions of trivial statements, then randomly arranging all of those trivial statements in one extremely long sequence.
Let Z be the conjunction of all these statements (randomly ordered as above).
Then Z is logically stronger than Y. But our subjective Bayesian cannot use Z to prove X (Finding Moby Dick in the Library of Babel is no easier than just writing Moby Dick oneself, and we’ve already assumed that our subject is unable to prove X under his own steam) whereas he can use Y to prove X.
If you say that then you’re conceding the point, because Y is nothing other than the conjunction of a carefully chosen subset of the trivial statements comprising Z, re-ordered so as to give a proof that can easily be followed.
Figuring out how to reorder them requires mathematical knowledge, a special kind of knowledge that can be generated, not just through contact with the external world, but through spending computer cycles on it.
Yes. It’s therefore important to quantify how many computer cycles and other resources are involved in computing a prior. There is a souped-up version of taw’s argument along those lines: either P = NP, or else every prior that is computable in polynomial time will fall for the conjunction fallacy.
If you want to make it a bit less unrealistic, imagine there are only, say, 1000 difficult proofs randomly chopped and spliced rather than a gazillion—but still too many for the subject to make head or tail of. Perhaps imagine them adding up to a book about the size of the Bible, which a person can memorize in its entirety given sufficient determination.
It’s not even guaranteed to be true (but you can verify that yourself much more easily than X directly).
Compare this with classical result of conjunction fallacy. In experiments people routinely believed that:
P(next year the Soviet Union will invade Poland, and the United States will break off diplomatic relations with the Soviet Union) > P(next year the United States will break off diplomatic relations with the Soviet Union).
Here X=the United States will break off diplomatic relations with the Soviet Union.
Y=the Soviet Union will invade Poland.
Wouldn’t your reasoning pretty much endorse what people were doing? (with Y—one possible scenario leading to X—being new information)
Hmmm, I now think the existence of Y is actually a distraction. The underlying process is:
produce estimate for P(X) ⇒ find proof of X ⇒ P(X) increases
If estimates are allowed to change in this manner, then of course they are allowed to change when someone else shows you a proof of X. (since P(X)=P(X) is also a law of probability) If they are not allowed to change in this manner, then subjective Bayesianism applied to mathematical laws collapses anyways.
From a purely human psychological perspective:
When someone tells me a proof of a theorem, it feels like I’ve learned something.
When I figure one out myself, it feels like I figured something out, as if I’d learned information through interacting with the natural world.
Are you meaning to tell me that no one learns anything in math class? Or they learn something, but the thing they learn isn’t information?
Caveat: Formalizing the concepts requires us to deviate from human experience sometimes. I don’t think the concept of information has been formalized, by Bayesians or Frequentists, in a way that deals with the problem of acting with limited computing time, aka the problem of logical uncertainty.
Would you agree that “P(X)” you’re describing is more like “some person’s answer when asked question X” than “probability of X”?
The main difference between two is that if “X” and “Y” are the same logical outcome, then their probabilities are necessarily the same, but an actual person can reply differently depending on how question was formulated.
If you’re interested in this subject, I recommend reading about epistemic modal logic—not necessarily for their solutions, but they’re clearly aware of this problem, and can describe it better than me.
You can get pwn’d if the person offering you the bets knows more than you. The only defense is to, when you’re uncertain, have bets such that you will not take X or you will not take -X (EDIT: Because you update on the information that they’re offering you a bet. I forgot that. Thanks JG.). This can be conceptualized as saying “I don’t know”
So yeah. If you suspect that someone may know more math than you, don’t take their bets about math.
Now, it’s possible to have someone pre-commit to not knowing stuff about the world. But they can’t pre-commit to not knowing stuff about math, or they can’t as easily.
You can get pwn’d if the person offering you the bets knows more than you. The only defense is to, when you’re uncertain, have bets such that you will not take X or you will not take -X. This can be conceptualized as saying “I don’t know”
Another defense is updating on the information that this person who knows more than you is offering this bet before you decide if you will accept it.
Given that they are presented at the same time (such as X is a conjecture, Y is a proof of the conjecture), yes, accepting these bets is being Dutch Booked. But upon seeing “X and Y” you would update “X” to something like 95%.
Given that they are presented in order (What bet do you take against X? Now that’s locked in, here is a proof Y. What bet do you take for “X and Y”?) this is a malady that all reasoners without complete information suffer from. I am not sure if that counts as a Dutch Book.
Given that they are presented in order [...] I am not sure if that counts as a Dutch Book.
It is trivial to reformulate this problem to X and X’ being logically equivalent, but not immediately noticeable as such, and a person being asked about X’ and (X and Y) or something like that.
Yes, but that sounds like “If you don’t take the time to check your logical equivalencies, you will take Dutch Books”. This is that same malady: it’s called being wrong. That is not a case of Bayesianism being open to Dutch Books: it is a case of wrong people being open to Dutch Books.
“If you don’t take the time to check your logical equivalencies, you will take Dutch Books”
You’re very wrong here.
By Goedel’s Incompleteness Theorem, there is no way to “take the time to check your logical equivalencies”. There are always things that are logically equivalent that your particular method of proving, no matter how sophisticated, will not find, in any amount of time.
This is somewhat specific to Bayesianism, as Bayesianism insists that you always give a definite numerical answer.
Not being able to refuse answering (by Bayesianism) + no guarantee of self-consistency (by Incompleteness) ⇒ Dutch booking
I admit defeat. When I am presented with enough unrefusable bets that incompleteness prevents me from realising are actually Dutch Books such that my utility falls consistently below some other method, I will swap to that method.
pr(0.5) that Collatz conjecture is true. My belief is synchronic (all mutually exclusive and exhaustive outcomes sum to 1) and diachronic (I will perform Bayesian updates on this belief correctly). I see no way to Dutch book me.
As for the koan;
Let “bet Y” = “Should Perfect Bayesian (PB) accept a bet (bet X) that PB will reject this bet?”
The koan, then, is:
Should PB accept bet X that PB will reject bet Y?
For payoffs of X less than 1 (bet a dollar to win fifty cents, say), PB should not take bet X. For payoffs of X greater than 1 (bet a dollar to receive a dollar fifty, say) PB should take bet X and then reject bet Y. For even payoffs, the opportunity cost of epsilon tells PB not to take the bet.
pr(0.5) that Collatz conjecture is true. My belief is synchronic (all mutually exclusive and exhaustive outcomes sum to 1) and diachronic (I will perform Bayesian updates on this belief correctly). I see no way to Dutch book me.
The trouble is that if the Collatz conjecture is true then {”Collatz conjecture is true”} constitutes an exhaustive set of outcomes, whose probability you think is only 1⁄2.
Should PB accept bet X that PB will reject bet Y?
I think the intended question is “Should PB accept bet X that PB will reject bet X?”
The trouble is that if the Collatz conjecture is true then {”Collatz conjecture is true”} constitutes an exhaustive set of outcomes, whose probability you think is only 1⁄2.
And if the Collatz conjecture is false then {”Collatz conjecture is false”} constitutes a single outcome, whose probability I think is 1⁄2. As of now I don’t know which premise (if true, if false) is actually the case, so I represent my uncertainty about the premises with a probability of 0.5. Representing uncertainty about an outcome (even an outcome that we would know if we were logically omniscient) does not make you dutch-bookable; incorrectly determining your uncertainty makes you dutch-bookable.
“Should PB accept bet X that PB will reject bet X?”
Even easier. No.
“I’ll bet you a dollar you won’t take this bet.”
“So if I take it, I lose a dollar, and if I don’t take it, I lose nothing? I reject the bet. Having rejected the bet, you will now proceed to point out that I would have won the bet. I will now proceed to show you that taking the bet closes off the path where I win the bet (it satisfies the failure condition), and rejecting the bet closes off the path where I win the bet (can’t win a bet you didn’t take), so my choices are between losing the bet and not taking the bet. Out of the only two options available to me, losing the bet costs me a dollar, not taking the bet costs me nothing. Quod erat demonstrandum.”
I am aware that my lack of logical omniscience is a problem for foundations of Bayesian epistemology (and, frankly, in the rest of my life too). The logical omniscience issue is another one I am interested in tackling, if you would like to write a post formulating that problem and maybe suggesting some paths for addressing it, by all means have at it. A set of ambiguous and somewhat rude questions directed at me or no one in particular is not a constructive way to discuss this issue.
Do you understand how any definite answer other than 0 and 1 means you just got successfully Dutch booked?
It doesn’t. Not even remotely. You clearly have no understanding of the Dutch book concept.
And this demonstrates the problem I have with “koans”. They tend to be used to try to make complete nonsense sound deep. The troubling thing is that it quite often works. The parent is upvoted still at the time of my reply and it almost certainly wouldn’t be if not for koan silliness.
Dear Perfect Bayesian,
What probability do you assign to Collatz conjecture being true? There’s plenty of other number theory statements I could ask about of course.
Do you understand how any definite answer other than 0 and 1 means you just got successfully Dutch booked?
A koan: Should Perfect Bayesian accept a bet that Perfect Bayesian will reject this bet?
The two are closely related.
Isn’t that like saying you got Dutch-booked for assigning 1⁄2 as the probability of heads (because a clairvoyant will be able to foresee that the coin is actually falling tails)?
The relevant point is that, in real life, computation requirements can keep you from calculating the exact Bayesian probability, which will lead to dutch-booking if an agent with enough computing power has a good model of the approximation you’re using.
Correct.
Not at all.
Collatz conjecture is true in every universe, or false in every universe.
You can slice it into a set of trivial statements which are trivially true or trivially false, like “Collatz conjecture is true for N=531” etc., connected by trivially true or trivially false logical statements.
There’s no way to meaningfully get any probability but 0 or 1 out of this, other than by claiming that some basic mathematical law is uncertain (and if you believe that, you are more Dutch bookable than entire Netherlands). I might not know how to Dutch book you yet, but logic dictates such a way exists.
Except thanks to Incompleteness Theorem, you have no way to find a definite answer to every such statement. No matter which strategy you choose, and how much time you have, you’ll either be inconsistent (Dutch bookable), or incomplete (not able to answer 0 or 1 - and as no other answer is valid, any answer you give makes you Dutch bookable).
Do you assign probability 1 to the proposition that 182547553 is prime? Right now, without doing an experiment on a calculator? (well, computer maybe. For most calculators testing this proposition would be somewhat tedious)
If yes, would you willing to pay me $10 if you ever found out it was not prime?
Conversely
Do you assign probability 0 to the proposition that 182547553 is prime? Right now, without doing an experiment on a calculator?
If yes, would you willing to pay me $10 if you ever found out it was prime?
EDIT: Actually, I suppose this counts as “doing it again”, even though I’m not Peter de Blanc. I think that makes me a bad bad person.
I suggest you look up the concept of “subjective Bayesian”. Probabilities are states of knowledge. If you don’t know an answer, it’s uncertain. If someone who doesn’t know anything you don’t can look over your odds and construct a knowably losing bet anyway, or construct a winning bet that you refuse, then you are Dutch-bookable.
Also, considering that you have apparently been reading this site for years and you still have not grasped the concept of subjective uncertainty, and you are still working with a frequentist notion of probability, nor yet have you even grasped the difference, I would suggest to you in all seriousness that you seek enlightenment elsewhere.
(Sorry, people, there’s got to be some point at which I can express that. Downvote if you must, I suppose, if you think such a concept is unspeakable or unallowed, but it’s not a judgment based on only one case of incomprehension, of course.)
I don’t know the background conflict. But at least one of taw’s points is correct. Any prior P, of any agent, has at least one of the following three properties:
It is not defined on all X—i.e. P is agnostic about some things
It has P(X) < P(X and Y) for at least one pair X and Y—i.e. P sometimes falls for the conjunction fallacy
It has P(X) = 1 for all mathematically provable statements X—i.e. P is an oracle.
You aren’t excused from having to pick one by rejecting frequentist theory.
To make use of a theory like probability one doesn’t have to have completely secure foundations. But it’s responsible to know what the foundational issues are. If you make a particularly dicey or weird application of probability theory, e.g. game theory with superintelligences, you should be prepared to explain (especially to yourself!) why you don’t expect those foundational issues to interfere with your argument.
About taw’s point in particular, I guess it’s possible that von Neumann gave a completely satisfactory solution when he was a teenager, or whatever, and that I’m just showcasing my ignorance. (I would dearly like to hear about this solution!) Otherwise your comment reads like you’re shooting the messenger.
Logical uncertainty is an open problem, of course (I attended a workshop on it once, and was surprised at how little progress has been made).
But so far as Dutch-booking goes, the obvious way out is 2 with the caveat that the probability distribution never has P(X) < (PX&Y) at the same time, i.e., you ask it P(X), it gives you an answer, you ask it P(X&Y), it gives you a higher answer, you ask it P(X) again and it now gives you an even higher answer from having updated its logical uncertainty upon seeing the new thought Y.
It is also clear from the above that taw does not comprehend the notion of subjective uncertainty, hence the notion of logical uncertainty.
Have any ideas?
Your full endorsement of evaporative cooling is quite disturbing.
Are you at least aware that epistemic position you’re promoting is highly contrarian?
How, exactly, to deal with logical uncertainty is an unsolved problem, no?
It’s not clear why it’s more of a problem for Bayesian than anyone else.
The post you linked to is a response to Eliezer arguing with Hanson, with Eliezer taking a pro-his-own-contrarianism stance. Of course he’s aware of that contrarianism.
Your choice is either accepting that you will be sometimes inconsistent, or accepting that you will sometimes answer “I don’t know” without providing a specific number, or both.
There’s nothing wrong with “I don’t know”.
For Perfect Bayesian or for Subjective Bayesian?
Subjective Bayesian does believe many statements of kind P(simple math step) = 1, P(X|conjunction of simple math steps) = 1, and yet P(X) < 1.
it does not believe math statements with probability 1 or 0 until it investigates them. As soon as it investigates whether (X|conjunction of simple math steps) is true and determines the answer, it sets P(X)=1.
The problem with “I don’t know” is that sometimes you have to make decisions. How do you propose to make decisions if you don’t know some relevant mathematical fact X?
For example, if you’re considering some kind of computer security system that is intended to last a long time, you really need an estimate for how likely it is that P=NP.
Then you need to fully accept that you will be inconsistent sometimes. And compartmentalize your belief system accordingly, or otherwise find a way to deal with these inconsistencies.
Well, they can wriggle out of this by denying P(simple math step) = 1, which is why I introduced this variation.
Doesn’t this imply you’d be willing to accept P(2+2=5) on good enough odds?
This might be pragmatically a reasonable thing to do, but if you accept that all math might be broken, you’ve already given up any hope of consistency.
If physics is deterministic then conditional on the state of the world at the time you make the bet, the probability of heads is either 0 or 1. The only disanalogy with your example is that you may not already have sufficient information to determine how the coin will land (which isn’t even a disanalogy if we assume that the person doesn’t know what the Collatz conjecture says). But suppose you did have that information—there would be vastly more of it than you could process in the time available, so it wouldn’t affect your probability assignments. (Note: The case where the Collatz conjecture turns out to be true but unprovable is analogous to the case where the laws of physics are deterministic but ‘uncomputable’ in some sense.)
Anyway, the real reason why I want to resist your line of argument here is due to Chaitin’s number “omega”, the “halting probability”. One can prove that the bits in the binary expansion of omega are algorithmically incompressible. More precisely: In order to deduce n bits’ worth of information about omega you need at least n—k bits’ worth of axioms, for some constant k. Hence, if you look sufficiently far along the binary expansion of omega, you find yourself looking at an infinite string of “random mathematical facts”. One “ought” to treat these numbers as having probability 1⁄2 of being 1 and 1⁄2 of being 0 (if playing games against opponents who lack oracle powers).
Deterministic world and “too much information to process” are uninteresting. All that simply means that due to practical constraints, sometimes the only reasonably thing is to assign no probability. As if we didn’t know that already. But probabilities still might be assignable in theory.
Except uncomputability means it won’t work even in theory. You are always Dutch bookable.
Chaitin’s number is not a mathematical entity—it’s creation of pure metaphysics.
The claim that kth bit of Chaitin’s number is 0 just doesn’t mean anything once k becomes big enough to include a procedure to compute Chaitin’s number.
Better: Sometimes the only reasonable thing is to assign a probability that’s strictly speaking “wrong”, but adequate if you’re only facing opponents who are (approximately) as hampered as you in terms of how much they know and much they can feasible compute. (E.g. Like humans playing poker, where the cards are only pseudo-random.)
If you want to say this is uninteresting, fine. I’m not trying to argue that it’s interesting.
Sorry, you’ve lost me.
Chaitin’s number is awfully tame by the standards of descriptive set theory. So what you’re really saying here is that you personally regard a whole branch of mathematics as “pure metaphysics”. Maybe a few philosophers of mathematics agree with you—I suspect most do not—but actual mathematicians will carry on studying mathematics regardless.
I’m not sure what you’re trying to say here but what you’ve actually written is false. Why do you think Chaitin’s number isn’t well defined?
For what it’s worth, I think you made a very interesting contribution to this thread, and I find it somewhat baffling that EY responded in the way he did (though perhaps there’s some ‘history’ here that I’m not aware of) and equally baffling that this has apparently caused others to downvote you.
No. I don’t understand that. Could you sketch the argument?
Incidentally, this may be playing with words, but the usual way of expressing the Dutch-book-victimhood of the hapless non-Bayesian agent is to say that a Dutch book can be constructed against him—not merely that a Dutch book exists. Successful gamblers engage in constructive, computable mathematics.
There is no hope for LessWrong as long as people keep conflating Perfect Bayesian and Subjective Bayesian.
Let’s take Subjective Bayesian first. The problem is—Subjective Bayesian breaks basic laws of probability as a routine matter.
Take the simplest possible law of probability P(X) >= P(X and Y).
Now let’s X be any mathematical theorem which you’re not sure about. 1 > P(X) > 0.
Let Y be some kind of “the following proof of X is correct”.
Verifying proofs is usually very simple, so very once you’re asked about P(X and Y), you can confidently reply P(X and Y) = 1. Y is not a new information about the world. It is usually conjunction of trivial statements which you already assigned probability 1.
That is—there’s infinite number of statements for which Subjective Bayesian will reply P(X) < P(X and Y).
For Subjective Bayesian X doesn’t even have to involve any infinities, just ask a simple question about cryptography which is pretty much guaranteed to be unsolvable before heat death of the universe, and you’re done.
At this point people far too often try to switch Perfect Bayesian for Subjective Bayesian.
And this is true, Perfect Bayesian wouldn’t make this particular mistake, and all probabilities of mathematical theorems he’d give would be 0 or 1, no exceptions. The problem is—Perfect Bayesians are not possible due to uncomputability.
If your version of Perfect Bayesian is computable, straightforward application of Rice Theorem shows he won’t be able to answer every question consistently.
If you claim some super-computable oracle version Perfect Bayesian—well, first that’s already metaphysics not mathematics, but in the end, this way of working around uncomputability does not work.
At any mention of uncomputability people far too often try to switch Subjective Bayesian for Perfect Bayesian (see Eliezer’s comment).
Excellent—now you’ve explained what you mean by “Perfect Bayesian” it all makes sense! (Though I can’t help thinking it would have saved time if you’d said this earlier.)
Still, I’m not keen on this redefinition of the word ‘metaphysics’, as though your philosophy of mathematics were ‘obviously correct’ ‘received wisdom’, when actually it’s highly contentious.
Anyway, I think this is a successful attack on a kind of “Bayesian absolutism” which claims that beings who (explicitly or implicitly) assign consistent probabilities to all expressible events, and update their beliefs in the Bayesian manner, can actually exist. That may be a straw man, though.
The obvious solution:
Y is information. It is not information about the world, but it is information—information about math.
I don’t think that works.
Imagine taking the proofs of all provable propositions that can be expressed in less than N characters (for some very large number N), writing them as conjunctions of trivial statements, then randomly arranging all of those trivial statements in one extremely long sequence.
Let Z be the conjunction of all these statements (randomly ordered as above).
Then Z is logically stronger than Y. But our subjective Bayesian cannot use Z to prove X (Finding Moby Dick in the Library of Babel is no easier than just writing Moby Dick oneself, and we’ve already assumed that our subject is unable to prove X under his own steam) whereas he can use Y to prove X.
The Bayesian doesn’t know Z is stronger than Y. He can’t even read all of Z. Or if you compress it, he can’t decompress it.
P(Y|Z)<1.
If you say that then you’re conceding the point, because Y is nothing other than the conjunction of a carefully chosen subset of the trivial statements comprising Z, re-ordered so as to give a proof that can easily be followed.
Figuring out how to reorder them requires mathematical knowledge, a special kind of knowledge that can be generated, not just through contact with the external world, but through spending computer cycles on it.
Yes. It’s therefore important to quantify how many computer cycles and other resources are involved in computing a prior. There is a souped-up version of taw’s argument along those lines: either P = NP, or else every prior that is computable in polynomial time will fall for the conjunction fallacy.
Imagine he has read and memorized all of Z.
If you want to make it a bit less unrealistic, imagine there are only, say, 1000 difficult proofs randomly chopped and spliced rather than a gazillion—but still too many for the subject to make head or tail of. Perhaps imagine them adding up to a book about the size of the Bible, which a person can memorize in its entirety given sufficient determination.
Oh I see. Chopped and spliced. That makes more sense. I missed that in your previous comment.
The Bayesian still does not know that Z implies Y, because he has not found Y in Z, so there still isn’t a problem.
In which sense is Y information?
It’s not even guaranteed to be true (but you can verify that yourself much more easily than X directly).
Compare this with classical result of conjunction fallacy. In experiments people routinely believed that:
P(next year the Soviet Union will invade Poland, and the United States will break off diplomatic relations with the Soviet Union) > P(next year the United States will break off diplomatic relations with the Soviet Union).
Here X=the United States will break off diplomatic relations with the Soviet Union. Y=the Soviet Union will invade Poland.
Wouldn’t your reasoning pretty much endorse what people were doing? (with Y—one possible scenario leading to X—being new information)
Hmmm, I now think the existence of Y is actually a distraction. The underlying process is:
produce estimate for P(X) ⇒ find proof of X ⇒ P(X) increases
If estimates are allowed to change in this manner, then of course they are allowed to change when someone else shows you a proof of X. (since P(X)=P(X) is also a law of probability) If they are not allowed to change in this manner, then subjective Bayesianism applied to mathematical laws collapses anyways.
From a purely human psychological perspective: When someone tells me a proof of a theorem, it feels like I’ve learned something. When I figure one out myself, it feels like I figured something out, as if I’d learned information through interacting with the natural world.
Are you meaning to tell me that no one learns anything in math class? Or they learn something, but the thing they learn isn’t information?
Caveat: Formalizing the concepts requires us to deviate from human experience sometimes. I don’t think the concept of information has been formalized, by Bayesians or Frequentists, in a way that deals with the problem of acting with limited computing time, aka the problem of logical uncertainty.
I think we almost agree already ;-)
Would you agree that “P(X)” you’re describing is more like “some person’s answer when asked question X” than “probability of X”?
The main difference between two is that if “X” and “Y” are the same logical outcome, then their probabilities are necessarily the same, but an actual person can reply differently depending on how question was formulated.
If you’re interested in this subject, I recommend reading about epistemic modal logic—not necessarily for their solutions, but they’re clearly aware of this problem, and can describe it better than me.
Ok, I understood that, but I still don’t see what it has to do with Dutch books.
P(X) < P(X and Y) gives you a dutch book.
OH I SEE. Revelation.
You can get pwn’d if the person offering you the bets knows more than you. The only defense is to, when you’re uncertain, have bets such that you will not take X or you will not take -X (EDIT: Because you update on the information that they’re offering you a bet. I forgot that. Thanks JG.). This can be conceptualized as saying “I don’t know”
So yeah. If you suspect that someone may know more math than you, don’t take their bets about math.
Now, it’s possible to have someone pre-commit to not knowing stuff about the world. But they can’t pre-commit to not knowing stuff about math, or they can’t as easily.
Another defense is updating on the information that this person who knows more than you is offering this bet before you decide if you will accept it.
That’s not the Dutch book I was talking about.
Let’s say you assign “X” probability of 50%, so you gladly take 60% bet against “X”.
But you assign “X and Y” probability 90%, so you as gladly take 80% bet for “X and Y”.
You just paid $1.20 for combinations of bets that can give you returns of at most $1 (or $0 if X turns out to be true but Y turns out to be false).
This is exactly a Dutch Book.
Given that they are presented at the same time (such as X is a conjecture, Y is a proof of the conjecture), yes, accepting these bets is being Dutch Booked. But upon seeing “X and Y” you would update “X” to something like 95%.
Given that they are presented in order (What bet do you take against X? Now that’s locked in, here is a proof Y. What bet do you take for “X and Y”?) this is a malady that all reasoners without complete information suffer from. I am not sure if that counts as a Dutch Book.
It is trivial to reformulate this problem to X and X’ being logically equivalent, but not immediately noticeable as such, and a person being asked about X’ and (X and Y) or something like that.
Yes, but that sounds like “If you don’t take the time to check your logical equivalencies, you will take Dutch Books”. This is that same malady: it’s called being wrong. That is not a case of Bayesianism being open to Dutch Books: it is a case of wrong people being open to Dutch Books.
You’re very wrong here.
By Goedel’s Incompleteness Theorem, there is no way to “take the time to check your logical equivalencies”. There are always things that are logically equivalent that your particular method of proving, no matter how sophisticated, will not find, in any amount of time.
This is somewhat specific to Bayesianism, as Bayesianism insists that you always give a definite numerical answer.
Not being able to refuse answering (by Bayesianism) + no guarantee of self-consistency (by Incompleteness) ⇒ Dutch booking
I admit defeat. When I am presented with enough unrefusable bets that incompleteness prevents me from realising are actually Dutch Books such that my utility falls consistently below some other method, I will swap to that method.
I’m curious.
pr(0.5) that Collatz conjecture is true. My belief is synchronic (all mutually exclusive and exhaustive outcomes sum to 1) and diachronic (I will perform Bayesian updates on this belief correctly). I see no way to Dutch book me.
As for the koan;
Let “bet Y” = “Should Perfect Bayesian (PB) accept a bet (bet X) that PB will reject this bet?”
The koan, then, is:
Should PB accept bet X that PB will reject bet Y?
For payoffs of X less than 1 (bet a dollar to win fifty cents, say), PB should not take bet X. For payoffs of X greater than 1 (bet a dollar to receive a dollar fifty, say) PB should take bet X and then reject bet Y. For even payoffs, the opportunity cost of epsilon tells PB not to take the bet.
The trouble is that if the Collatz conjecture is true then {”Collatz conjecture is true”} constitutes an exhaustive set of outcomes, whose probability you think is only 1⁄2.
I think the intended question is “Should PB accept bet X that PB will reject bet X?”
And if the Collatz conjecture is false then {”Collatz conjecture is false”} constitutes a single outcome, whose probability I think is 1⁄2. As of now I don’t know which premise (if true, if false) is actually the case, so I represent my uncertainty about the premises with a probability of 0.5. Representing uncertainty about an outcome (even an outcome that we would know if we were logically omniscient) does not make you dutch-bookable; incorrectly determining your uncertainty makes you dutch-bookable.
Even easier. No. “I’ll bet you a dollar you won’t take this bet.” “So if I take it, I lose a dollar, and if I don’t take it, I lose nothing? I reject the bet. Having rejected the bet, you will now proceed to point out that I would have won the bet. I will now proceed to show you that taking the bet closes off the path where I win the bet (it satisfies the failure condition), and rejecting the bet closes off the path where I win the bet (can’t win a bet you didn’t take), so my choices are between losing the bet and not taking the bet. Out of the only two options available to me, losing the bet costs me a dollar, not taking the bet costs me nothing. Quod erat demonstrandum.”
I am aware that my lack of logical omniscience is a problem for foundations of Bayesian epistemology (and, frankly, in the rest of my life too). The logical omniscience issue is another one I am interested in tackling, if you would like to write a post formulating that problem and maybe suggesting some paths for addressing it, by all means have at it. A set of ambiguous and somewhat rude questions directed at me or no one in particular is not a constructive way to discuss this issue.
I recommend starting here.
Lesswrongian-Bayesian P works as this kind of an epistemic modal operator, and has exactly the problems I mentioned.
It doesn’t. Not even remotely. You clearly have no understanding of the Dutch book concept.
And this demonstrates the problem I have with “koans”. They tend to be used to try to make complete nonsense sound deep. The troubling thing is that it quite often works. The parent is upvoted still at the time of my reply and it almost certainly wouldn’t be if not for koan silliness.