I may be misunderstanding the intended scope of the post, but currently the argument reads to me more like a critique of some probabilistic frameworks than a critique of probabilistic reasoning in general.
Epistemic status: similar to author, most prior work I read is scattered across many, often very confusingly written blog posts, and I can’t easily tell where I first came across various ideas. I tried to focus on “general” deductive logic based on my reading of the post (which may be wrong) instead of applying stuff that is too framework-specific.
I will also provide feedback on some wording (seems like author tried too hard to make post streamlined and/or conform to style norms.)
This first post will look at some possible definitions of probabilities and why I think they don’t really work
Don’t really work for what, more exactly? Descriptive account of how agents (humans) interpret probability? Account of how probability should be conceptually seen? How we should use probability for epistemic rationality? Instrumental rationality? It seems that you want something for epistemic and instrumental rationality based on the rest of the post, but I think it would be better if you clarified from the start.
What do I mean when I say that I give a 10% probability that it’s going to rain in my town tomorrow? This 10% probability doesn’t refer to any tangible fact about the real world. Sure, there is some amount of objective randomness in whether it will rain or not tomorrow, due to quantum randomness. But I have no idea how big the quantum effects are on the weather tomorrow, and when I say I give a 10% chance for rain, I’m clearly not referring to the true quantum probabilities.
I’m not sure if this is a general problem? If A told me their probability, the following interpretation seems reasonable: “According to A’s computations, which probably weren’t at a level of precision/detail involving quantum uncertainty, in 10% of their predicted worlds rain would happen tomorrow in my town.)”.
Formal form: , where refers to A’s measure function (based on close worlds etc.). This seems pretty intuitive, I’m also not sure if the frequentist critique is that relevant. I’m unsure whether frequentists wanted to generalize work based on based instances to everything, including the Russian invasion of Ukraine. Some space could be dedicated to other frameworks (the one above is based on my intuition).
The classical Bayesian view holds that probabilities are just my subjective credences; they only live in my head. I find this view appealing. Still, if someone tells me he thinks there is a 50% chance that Bigfoot is standing in the next room, I wouldn’t just shrug and say “Yep, it’s all subjective, like liking chocolate and vanilla ice cream. He says 50%, that’s as good as any other probability estimate.”
I assume you wanted to move quickly to the Solomonoff induction part, but that is not sufficient evidence against general bayesianism. Dismisssing it via an example of aburd priors, which most Bayesians would disagree with and (presumably) try to fix from inside the framework, is also suspicious.
On Solomonoff priors producing unintuive results:
I don’t see how being unintuitive compared to a naive conception of probability is evidence against probability-in-general/Solomonoff induction, instead of being evidence either the naive conception of probability being bad or some assumption being false. In fact, lacking other adequate explanations for the (apparent) plausibility other worlds being solipsistic simulations, I would be rationally be forced to take them into consideration and, implicitly, heighten my credence in the Solomonoff prior, if your link to them is assumed.[1]
This makes me think that defining probabilities based on a formal prior is not a very useful concept, and doesn’t really match how we normally think about probabilities.
I agree that formal priors like Solomonoff induction are bad (or at least incomplete). However, you are forced to base your theory on some priors (more exactly, priors derived from biology). I don’t think priors being a formal component would make a hypothetical theory “worse”, or that lacking formal priors would make a theory intrinsically better.
Also, basing your logic solely on the failures of Solomonoff priors is not valid by itself[2]. Why wouldn’t the conclusion be something like “I don’t know”/”We don’t know”? In general, it seems to me that your attempts to make it streamlined makes it feel like the post is overly focused on defeating a selection of theories with well-known flaws and implying “ergo, only this framework can save us”[3] instead of accepting uncertainty.
For most confusing philosophical questions, I think the best way to get out of the definitional quagmire is to try to form the questions in a way that is action-relevant. If I need to make an actual decision in a (possibly hypothetical) situation, that often clarifies my thinking, and dissolves the semantic squabbles that were irrelevant to the main question
In the case of probabilities, I think it’s often best to think of them as the betting odds at which I’d be indifferent between betting in either direction.
In regards to your proposed solution for operationalization: Why is winning hypothetical bets action-relevant? Why would an agent want to calibrate their probabilities of X via optimizing bets specifically on money, chocolate, or hypothetical terminal values? It seems like redundant mental gymnastics to get an equivalent result at best. At most, you can imagine hypothetical bets where you get fixed utility instead of chocolates etc., but you can equivalently frame that as optimizing a number instead of gambling.
I personally haven’t been blocked by such definitional quagmires from directly calculating probabilities and expected utilities. I think that the framing it as “this philosophical problem highlights a possible bias/irrational aspect in my calculation algorithm” is better than the “These paradoxes prove that the concept of probability is incoherent/not useful”.
Some people don’t like these betting-based definitions, and insist that there must be something more real in probabilities than just how one would bet.[10] I will write more about this in a future post, but for now I will just say that I’m myself very sympathetic to thinking in terms of bets. I believe basically everything can be formulated as a “bet”, and I don’t quite see what could be there about probabilities that can’t be phrased this way.
Stuff that can’t be phrased this way: the definition; conceptual clarity. “Winning hypothetical bets” and “measure of how likely something is” seem conceptually distinct, even if you can apply bets for equivalent results.
For philosophically confusing questions involving anthropics and the simulation hypothesis, I refuse to answer with probabilities and instead ask what exact bet we are hypothetically making, or what action we need to decide on. This makes me reluctant to pick a side in the SIA vs SSA debate in anthropics; I just don’t believe it’s the right level of abstraction to ask these questions. (Though SIA is generally closer to the mark in my opinion.)
Isn’t the “bet” in such problems implied to be simply the “truth”/”best representation of (some part of) reality under the conditions of the problem”? Probabilities abstract utilities away, yes, but are also work for every utility function.
Forcing probability discussions into stuff like “if the Sleeping Beauty woke up and got the utility equivalent of a chocolate under [betting conditions]” is logically suspicious (why are conceptions of probability other than bets not explored, more precisely? The post seems to assume by default bets are better.).
On probabilities for infinitesimal/supernatural scenarios, including pascalian wagers: using hyperreals or surreal probabilities works mathematically, and arguably still count as “natural notions” of probability. I don’t know how common this position is, or how to calibrate those infinitesimals specifically[4], but I think it merits consideration.
So I will need some method to weigh against each other the consequences of my actions in infinite possible worlds. I will write more about my proposed solutions in my next posts, but I believe that probabilities are not the right abstraction to handle these questions in general.
I kind of agree with the conclusion, but as a kind of lemma based on properties of expectationalism. You can ignore the probabilities of nihilistic worlds or other “decisionally irrelevant” stuff, simply because the utilities would be forced to be 0. You are also right that ultimately we want to make decisions, but this post hasn’t convinced me why one should abolish probability-weighed expectationalism to determine the right action instead of using computational tricks or refining the framework.[5]
I personally don’t think that “Current technological abilities imply we likely live in nihilistic simulations” is that positively correlated specifically with Solomonoff priors, but I may be wrong.
As you implied, it’s enough to calibrate enough in an action-relevant way (e.g. “whether to follow the wager”), though I consider that to be more of a computational trick.
I may be misunderstanding the intended scope of the post, but currently the argument reads to me more like a critique of some probabilistic frameworks than a critique of probabilistic reasoning in general.
Epistemic status: similar to author, most prior work I read is scattered across many, often very confusingly written blog posts, and I can’t easily tell where I first came across various ideas. I tried to focus on “general” deductive logic based on my reading of the post (which may be wrong) instead of applying stuff that is too framework-specific.
I will also provide feedback on some wording (seems like author tried too hard to make post streamlined and/or conform to style norms.)
Don’t really work for what, more exactly? Descriptive account of how agents (humans) interpret probability? Account of how probability should be conceptually seen? How we should use probability for epistemic rationality? Instrumental rationality? It seems that you want something for epistemic and instrumental rationality based on the rest of the post, but I think it would be better if you clarified from the start.
I’m not sure if this is a general problem? If A told me their probability, the following interpretation seems reasonable: “According to A’s computations, which probably weren’t at a level of precision/detail involving quantum uncertainty, in 10% of their predicted worlds rain would happen tomorrow in my town.)”.
Formal form:
I’m also not sure if the frequentist critique is that relevant. I’m unsure whether frequentists wanted to generalize work based on based instances to everything, including the Russian invasion of Ukraine. Some space could be dedicated to other frameworks (the one above is based on my intuition).
I assume you wanted to move quickly to the Solomonoff induction part, but that is not sufficient evidence against general bayesianism. Dismisssing it via an example of aburd priors, which most Bayesians would disagree with and (presumably) try to fix from inside the framework, is also suspicious.
On Solomonoff priors producing unintuive results:
I don’t see how being unintuitive compared to a naive conception of probability is evidence against probability-in-general/Solomonoff induction, instead of being evidence either the naive conception of probability being bad or some assumption being false. In fact, lacking other adequate explanations for the (apparent) plausibility other worlds being solipsistic simulations, I would be rationally be forced to take them into consideration and, implicitly, heighten my credence in the Solomonoff prior, if your link to them is assumed.[1]
I agree that formal priors like Solomonoff induction are bad (or at least incomplete). However, you are forced to base your theory on some priors (more exactly, priors derived from biology). I don’t think priors being a formal component would make a hypothetical theory “worse”, or that lacking formal priors would make a theory intrinsically better.
Also, basing your logic solely on the failures of Solomonoff priors is not valid by itself[2]. Why wouldn’t the conclusion be something like “I don’t know”/”We don’t know”? In general, it seems to me that your attempts to make it streamlined makes it feel like the post is overly focused on defeating a selection of theories with well-known flaws and implying “ergo, only this framework can save us”[3] instead of accepting uncertainty.
In regards to your proposed solution for operationalization: Why is winning hypothetical bets action-relevant? Why would an agent want to calibrate their probabilities of X via optimizing bets specifically on money, chocolate, or hypothetical terminal values? It seems like redundant mental gymnastics to get an equivalent result at best. At most, you can imagine hypothetical bets where you get fixed utility instead of chocolates etc., but you can equivalently frame that as optimizing a number instead of gambling.
I personally haven’t been blocked by such definitional quagmires from directly calculating probabilities and expected utilities. I think that the framing it as “this philosophical problem highlights a possible bias/irrational aspect in my calculation algorithm” is better than the “These paradoxes prove that the concept of probability is incoherent/not useful”.
Stuff that can’t be phrased this way: the definition; conceptual clarity. “Winning hypothetical bets” and “measure of how likely something is” seem conceptually distinct, even if you can apply bets for equivalent results.
Isn’t the “bet” in such problems implied to be simply the “truth”/”best representation of (some part of) reality under the conditions of the problem”? Probabilities abstract utilities away, yes, but are also work for every utility function.
Forcing probability discussions into stuff like “if the Sleeping Beauty woke up and got the utility equivalent of a chocolate under [betting conditions]” is logically suspicious (why are conceptions of probability other than bets not explored, more precisely? The post seems to assume by default bets are better.).
On probabilities for infinitesimal/supernatural scenarios, including pascalian wagers: using hyperreals or surreal probabilities works mathematically, and arguably still count as “natural notions” of probability. I don’t know how common this position is, or how to calibrate those infinitesimals specifically[4], but I think it merits consideration.
I kind of agree with the conclusion, but as a kind of lemma based on properties of expectationalism. You can ignore the probabilities of nihilistic worlds or other “decisionally irrelevant” stuff, simply because the utilities would be forced to be 0. You are also right that ultimately we want to make decisions, but this post hasn’t convinced me why one should abolish probability-weighed expectationalism to determine the right action instead of using computational tricks or refining the framework.[5]
I personally don’t think that “Current technological abilities imply we likely live in nihilistic simulations” is that positively correlated specifically with Solomonoff priors, but I may be wrong.
I think you agree with your “This makes me think” hedging, but I wanted to point it out explicitly.
Sorry for exaggerated phrasing.
As you implied, it’s enough to calibrate enough in an action-relevant way (e.g. “whether to follow the wager”), though I consider that to be more of a computational trick.
By the way, the way you phrased it in conclusions made it too similar to the fallacious “Abstractions are too weak for real phenomena” for my liking.