“(I haven’t downvoted this question nor any of Haziq’s others; but my guess is that this one was downvoted because it’s only a question worth asking if Halpern’s counterexample to Cox’s theorem is a serious problem, which johnswentworth already gave very good reasons for thinking it isn’t in response to one of Haziq’s other questions; so readers may reasonably wonder whether he’s actually paying any attention to the answers his questions get. Haziq did engage with johnswentworth in that other question—but from this question you’d never guess that any of that had happened.)”
Sorry, haven’t checked LW in a while. I actually came across this comment when I was trying to delete my LW account due to the “shoot the messenger” phenomenon that TAG was describing.
I do not think that Johnwentworth’s answer is satisfactory. In his response to my previous question, he claims that Cox’s theorem holds under very specific conditions which doesn’t happen in most cases. He also claims that probability as extended logic is justified by empirical evidence. I don’t think this is a good justification unless he happens to have an ACME plausibility-o-meter.
David Chapman, another messenger you (meaning LW) were too quick to shoot, explains the issues with subjective Bayesianism here:
I do agree that this framework is useful but only in the same sense that frequentism is useful. I consider myself a ”pragmatic statistician “ who doesn’t hesitate to use frequentist or Bayesian methods, as long as they are useful, because the justifications for either seem to be equally worse.
‘It might turn out that the way Cox’s theorem is wrong is that the requirements it imposes for a minimally-reasonable belief system need strengthening, but in ways that we would regard as reasonable. In that case there would still be a theorem along the lines of “any reasonable way of structuring your beliefs is equivalent to probability theory with Bayesian updates”.’
I find this statement to be quite disturbing because it seems to me that you are assuming Jaynes-Cox theory to be true first and then trying to find a proof for it. Sounds very much like confirmation bias. Van Horn’s paper could potentially revive Cox’s theorem but nobody’s talking about it because they are not ready to accept that Cox’s theorem has any issues in the first place.
I think the messenger-shooting is quite apparent in LW. It’s the reason why posts that oppose or criticise the “tenets of LW”, that LW members adhere to in a cult-like fashion, are so scarce. For instance, Chapman’s critique of LW seems to have been ignored altogether.
“The most dangerous ideas in a society are not the ones being argued, but the ones that are assumed.”
I guess there’s not that much point responding to this, since Haziq has apparently now deleted his account, but it seems worth saying a few words.
Haziq says he’s deleting his account because of LW’s alleged messenger-shooting, but I don’t see any sign that he was ever “shot” in any sense beyond this: one of his several questions received a couple of downvotes.
What johnswentworth’s answer about Cox’s theorem says isn’t at all that it “holds under very specific conditions which doesn’t happen in most cases”.
You’ll get no objection from me to the idea of being a “pragmatic statistician”.
No, I am not at all “assuming Jaynes-Cox theory to be true first and then trying to find a proof for it”. I am saying: the specific scenario you describe (there’s a hole in the proof of Cox’s theorem) might play out in various ways and here are some of them. Some of them would mean something a bit like “Less Wrong is dead” (though, I claim, not exactly that); some of them wouldn’t. I mentioned some of both.
I can’t speak for anyone else here, but for me Cox’s theorem isn’t foundational in the sort of way it sounds as if you think it is (or should be?). If Cox’s theorem turns out to be disastrously wrong, that would be very interesting, but rather little of my thinking depends on Cox’s theorem. It’s a bit as if you went up to a Christian and said “Is Christianity dead if the ontological argument is invalid?”; most Christians aren’t Christians because they were persuaded by the ontological argument, and I think most Bayesians (in the sense in which LW folks are mostly Bayesians) aren’t Bayesians because they were persuaded by Cox’s theorem.
I do not know what it would mean to adhere to, say, Cox’s theorem “in a cult-like fashion”.
The second-last bullet point there is maybe the most important, and warrants a bit more explanation.
Whether anything “similar enough” to Cox’s theorem is true or not, the following things are (I think) rather uncontroversial:
We should hold most (maybe all) of our opinions with some degree of uncertainty.
One possible way of thinking about opinions-with-uncertainty is as probabilities.
If we think about our beliefs this way, then there are some theorems telling us how to adjust them in the light of new evidence, how our beliefs about various logically-related propositions should be related, etc.
No obviously-better general approach to quantifying strength of beliefs is known.
To be clear, this doesn’t mean that nothing is known that is ever better at anything than probability theory with Bayesian updates. E.g., “provably approximately correct learning” isn’t (so far as I know) equivalent to anything Bayesian, and it gives some nice guarantees that (so far as I know) no Bayesian approach is known to give. So when what you want is what PACL theory gives you, you should be using PACL.
For me, these are sufficient to justify a generally-Bayes-flavoured approach, by which I mean:
Of course I don’t literally attach numerical probabilities to all my beliefs. Nor do I think it’s obvious that any real reasoner, given the finite resources we inevitably have, should be explicitly probabilistic about everything.
But if for some reason I need to think clearly about how plausible I find various possibilities, I generally do it in terms of probabilities. (Taking care, e.g., to notice when there is danger of double-counting things, which is an easy way to go wrong when applying Bayesian probability naively.)
If I notice that some element of how I think is outright inconsistent with this way of quantifying uncertainty, consider whether it’s mistaken (albeit possibly a useful approximation). E.g., most people’s intuitive judgements of how likely things are produce instances of the (poorly named) “conjunction fallacy”, plausibly often because we tend to apply the “representativeness heuristic”; on reflection I think this genuinely does indicate places where our intuitive judgements are mistaken, and trying to notice it happening and do something cleverer is of some value.
(None of this feels very cultish to me.)
Finding a bug in the proof of Cox’s theorem doesn’t do anything to invalidate any of the above. Finding a concrete case of a structure other than probabilities-with-Bayesian-updates does better (in some sense of “better”) on a problem resembling actual real-world reasoning absolutely might; in particular, it might make it false that “no obviously-better general approach to quantifying strength of beliefs is known”. Halpern’s counterexample to Cox is not, so far as I can tell, like that; it depends essentially on a sort of “sparsity” that doesn’t hold if what you’re trying to assign credences to is all propositions you might consider about how the world is. I think (indeed, I think Halpern pointed this out, but I may be misremembering) you can fix up the proof by adding an assumption saying that this sort of sparsity doesn’t occur, and although that assumption is ugly and technical I think it is a reasonable assumption in real life; and in the most obvious regime where you can’t make that assumption—where everything is finite—the van Horn approach seems to yield essentially the same conclusions with a pretty modest set of assumptions that are reasonable there.
So far as I can tell by introspection, I’m not saying all this because I am determined not to admit the possibility that Cox’s theorem might be all wrong. It looks to me like it isn’t all wrong, because I’ve looked at the alleged issues and some alleged patches for them and I think the patches work and the issues are purely technical. But it could be all wrong, and that possibility feels more interesting than upsetting to me.
“(I haven’t downvoted this question nor any of Haziq’s others; but my guess is that this one was downvoted because it’s only a question worth asking if Halpern’s counterexample to Cox’s theorem is a serious problem, which johnswentworth already gave very good reasons for thinking it isn’t in response to one of Haziq’s other questions; so readers may reasonably wonder whether he’s actually paying any attention to the answers his questions get. Haziq did engage with johnswentworth in that other question—but from this question you’d never guess that any of that had happened.)”
Sorry, haven’t checked LW in a while. I actually came across this comment when I was trying to delete my LW account due to the “shoot the messenger” phenomenon that TAG was describing.
I do not think that Johnwentworth’s answer is satisfactory. In his response to my previous question, he claims that Cox’s theorem holds under very specific conditions which doesn’t happen in most cases. He also claims that probability as extended logic is justified by empirical evidence. I don’t think this is a good justification unless he happens to have an ACME plausibility-o-meter.
David Chapman, another messenger you (meaning LW) were too quick to shoot, explains the issues with subjective Bayesianism here:
https://metarationality.com/probabilism-applicability
https://metarationality.com/probability-limitations
I do agree that this framework is useful but only in the same sense that frequentism is useful. I consider myself a ”pragmatic statistician “ who doesn’t hesitate to use frequentist or Bayesian methods, as long as they are useful, because the justifications for either seem to be equally worse.
‘It might turn out that the way Cox’s theorem is wrong is that the requirements it imposes for a minimally-reasonable belief system need strengthening, but in ways that we would regard as reasonable. In that case there would still be a theorem along the lines of “any reasonable way of structuring your beliefs is equivalent to probability theory with Bayesian updates”.’
I find this statement to be quite disturbing because it seems to me that you are assuming Jaynes-Cox theory to be true first and then trying to find a proof for it. Sounds very much like confirmation bias. Van Horn’s paper could potentially revive Cox’s theorem but nobody’s talking about it because they are not ready to accept that Cox’s theorem has any issues in the first place.
I think the messenger-shooting is quite apparent in LW. It’s the reason why posts that oppose or criticise the “tenets of LW”, that LW members adhere to in a cult-like fashion, are so scarce. For instance, Chapman’s critique of LW seems to have been ignored altogether.
“The most dangerous ideas in a society are not the ones being argued, but the ones that are assumed.”
— C. S. Lewis
I guess there’s not that much point responding to this, since Haziq has apparently now deleted his account, but it seems worth saying a few words.
Haziq says he’s deleting his account because of LW’s alleged messenger-shooting, but I don’t see any sign that he was ever “shot” in any sense beyond this: one of his several questions received a couple of downvotes.
What johnswentworth’s answer about Cox’s theorem says isn’t at all that it “holds under very specific conditions which doesn’t happen in most cases”.
You’ll get no objection from me to the idea of being a “pragmatic statistician”.
No, I am not at all “assuming Jaynes-Cox theory to be true first and then trying to find a proof for it”. I am saying: the specific scenario you describe (there’s a hole in the proof of Cox’s theorem) might play out in various ways and here are some of them. Some of them would mean something a bit like “Less Wrong is dead” (though, I claim, not exactly that); some of them wouldn’t. I mentioned some of both.
I can’t speak for anyone else here, but for me Cox’s theorem isn’t foundational in the sort of way it sounds as if you think it is (or should be?). If Cox’s theorem turns out to be disastrously wrong, that would be very interesting, but rather little of my thinking depends on Cox’s theorem. It’s a bit as if you went up to a Christian and said “Is Christianity dead if the ontological argument is invalid?”; most Christians aren’t Christians because they were persuaded by the ontological argument, and I think most Bayesians (in the sense in which LW folks are mostly Bayesians) aren’t Bayesians because they were persuaded by Cox’s theorem.
I do not know what it would mean to adhere to, say, Cox’s theorem “in a cult-like fashion”.
The second-last bullet point there is maybe the most important, and warrants a bit more explanation.
Whether anything “similar enough” to Cox’s theorem is true or not, the following things are (I think) rather uncontroversial:
We should hold most (maybe all) of our opinions with some degree of uncertainty.
One possible way of thinking about opinions-with-uncertainty is as probabilities.
If we think about our beliefs this way, then there are some theorems telling us how to adjust them in the light of new evidence, how our beliefs about various logically-related propositions should be related, etc.
No obviously-better general approach to quantifying strength of beliefs is known.
To be clear, this doesn’t mean that nothing is known that is ever better at anything than probability theory with Bayesian updates. E.g., “provably approximately correct learning” isn’t (so far as I know) equivalent to anything Bayesian, and it gives some nice guarantees that (so far as I know) no Bayesian approach is known to give. So when what you want is what PACL theory gives you, you should be using PACL.
For me, these are sufficient to justify a generally-Bayes-flavoured approach, by which I mean:
Of course I don’t literally attach numerical probabilities to all my beliefs. Nor do I think it’s obvious that any real reasoner, given the finite resources we inevitably have, should be explicitly probabilistic about everything.
But if for some reason I need to think clearly about how plausible I find various possibilities, I generally do it in terms of probabilities. (Taking care, e.g., to notice when there is danger of double-counting things, which is an easy way to go wrong when applying Bayesian probability naively.)
If I notice that some element of how I think is outright inconsistent with this way of quantifying uncertainty, consider whether it’s mistaken (albeit possibly a useful approximation). E.g., most people’s intuitive judgements of how likely things are produce instances of the (poorly named) “conjunction fallacy”, plausibly often because we tend to apply the “representativeness heuristic”; on reflection I think this genuinely does indicate places where our intuitive judgements are mistaken, and trying to notice it happening and do something cleverer is of some value.
(None of this feels very cultish to me.)
Finding a bug in the proof of Cox’s theorem doesn’t do anything to invalidate any of the above. Finding a concrete case of a structure other than probabilities-with-Bayesian-updates does better (in some sense of “better”) on a problem resembling actual real-world reasoning absolutely might; in particular, it might make it false that “no obviously-better general approach to quantifying strength of beliefs is known”. Halpern’s counterexample to Cox is not, so far as I can tell, like that; it depends essentially on a sort of “sparsity” that doesn’t hold if what you’re trying to assign credences to is all propositions you might consider about how the world is. I think (indeed, I think Halpern pointed this out, but I may be misremembering) you can fix up the proof by adding an assumption saying that this sort of sparsity doesn’t occur, and although that assumption is ugly and technical I think it is a reasonable assumption in real life; and in the most obvious regime where you can’t make that assumption—where everything is finite—the van Horn approach seems to yield essentially the same conclusions with a pretty modest set of assumptions that are reasonable there.
So far as I can tell by introspection, I’m not saying all this because I am determined not to admit the possibility that Cox’s theorem might be all wrong. It looks to me like it isn’t all wrong, because I’ve looked at the alleged issues and some alleged patches for them and I think the patches work and the issues are purely technical. But it could be all wrong, and that possibility feels more interesting than upsetting to me.