I think this phenomenon can be likened to strawmanning, since both include defense against an imagined version of the “actual meaning”. More exactly, I think it can be considered an instance of “subtext strawmanning”, since it probably came from applying exaggerations to the connotation of the criticism, using logic like “criticism ⇒ impolite ⇒ disrespectful ⇒ threatening ⇒ actual danger”.
In general, paying attention to the way in which parties interpret fallaciously aspects of a discussion other than the actual logic seems like a useful thing to do.
What’s going on in someone’s head when they jump from “it’s impossible to avoid giving offense when delivering serious criticism” to “but we can at least achieve some basic level of mutual respect about whether other people deserve to live”?
This seems related to the “Argument is War” class of metaphors. They may have (subconsciously) thought that the consequences of war (i.e., danger) also apply to things like blogging, since the criticizing person has “attacked” their argument/post through their criticism. While fallacious, I don’t think such logic is absurd or implausible.
I may be misunderstanding the intended scope of the post, but currently the argument reads to me more like a critique of some probabilistic frameworks than a critique of probabilistic reasoning in general.
Epistemic status: similar to author, most prior work I read is scattered across many, often very confusingly written blog posts, and I can’t easily tell where I first came across various ideas. I tried to focus on “general” deductive logic based on my reading of the post (which may be wrong) instead of applying stuff that is too framework-specific.
I will also provide feedback on some wording (seems like author tried too hard to make post streamlined and/or conform to style norms.)
Don’t really work for what, more exactly? Descriptive account of how agents (humans) interpret probability? Account of how probability should be conceptually seen? How we should use probability for epistemic rationality? Instrumental rationality? It seems that you want something for epistemic and instrumental rationality based on the rest of the post, but I think it would be better if you clarified from the start.
I’m not sure if this is a general problem? If A told me their probability, the following interpretation seems reasonable: “According to A’s computations, which probably weren’t at a level of precision/detail involving quantum uncertainty, in 10% of their predicted worlds rain would happen tomorrow in my town.)”.
Formal form:
I’m also not sure if the frequentist critique is that relevant. I’m unsure whether frequentists wanted to generalize work based on based instances to everything, including the Russian invasion of Ukraine. Some space could be dedicated to other frameworks (the one above is based on my intuition).
I assume you wanted to move quickly to the Solomonoff induction part, but that is not sufficient evidence against general bayesianism. Dismisssing it via an example of aburd priors, which most Bayesians would disagree with and (presumably) try to fix from inside the framework, is also suspicious.
On Solomonoff priors producing unintuive results:
I don’t see how being unintuitive compared to a naive conception of probability is evidence against probability-in-general/Solomonoff induction, instead of being evidence either the naive conception of probability being bad or some assumption being false. In fact, lacking other adequate explanations for the (apparent) plausibility other worlds being solipsistic simulations, I would be rationally be forced to take them into consideration and, implicitly, heighten my credence in the Solomonoff prior, if your link to them is assumed.[1]
I agree that formal priors like Solomonoff induction are bad (or at least incomplete). However, you are forced to base your theory on some priors (more exactly, priors derived from biology). I don’t think priors being a formal component would make a hypothetical theory “worse”, or that lacking formal priors would make a theory intrinsically better.
Also, basing your logic solely on the failures of Solomonoff priors is not valid by itself[2]. Why wouldn’t the conclusion be something like “I don’t know”/”We don’t know”? In general, it seems to me that your attempts to make it streamlined makes it feel like the post is overly focused on defeating a selection of theories with well-known flaws and implying “ergo, only this framework can save us”[3] instead of accepting uncertainty.
In regards to your proposed solution for operationalization: Why is winning hypothetical bets action-relevant? Why would an agent want to calibrate their probabilities of X via optimizing bets specifically on money, chocolate, or hypothetical terminal values? It seems like redundant mental gymnastics to get an equivalent result at best. At most, you can imagine hypothetical bets where you get fixed utility instead of chocolates etc., but you can equivalently frame that as optimizing a number instead of gambling.
I personally haven’t been blocked by such definitional quagmires from directly calculating probabilities and expected utilities. I think that the framing it as “this philosophical problem highlights a possible bias/irrational aspect in my calculation algorithm” is better than the “These paradoxes prove that the concept of probability is incoherent/not useful”.
Stuff that can’t be phrased this way: the definition; conceptual clarity. “Winning hypothetical bets” and “measure of how likely something is” seem conceptually distinct, even if you can apply bets for equivalent results.
Isn’t the “bet” in such problems implied to be simply the “truth”/”best representation of (some part of) reality under the conditions of the problem”? Probabilities abstract utilities away, yes, but are also work for every utility function.
Forcing probability discussions into stuff like “if the Sleeping Beauty woke up and got the utility equivalent of a chocolate under [betting conditions]” is logically suspicious (why are conceptions of probability other than bets not explored, more precisely? The post seems to assume by default bets are better.).
On probabilities for infinitesimal/supernatural scenarios, including pascalian wagers: using hyperreals or surreal probabilities works mathematically, and arguably still count as “natural notions” of probability. I don’t know how common this position is, or how to calibrate those infinitesimals specifically[4], but I think it merits consideration.
I kind of agree with the conclusion, but as a kind of lemma based on properties of expectationalism. You can ignore the probabilities of nihilistic worlds or other “decisionally irrelevant” stuff, simply because the utilities would be forced to be 0. You are also right that ultimately we want to make decisions, but this post hasn’t convinced me why one should abolish probability-weighed expectationalism to determine the right action instead of using computational tricks or refining the framework.[5]
I personally don’t think that “Current technological abilities imply we likely live in nihilistic simulations” is that positively correlated specifically with Solomonoff priors, but I may be wrong.
I think you agree with your “This makes me think” hedging, but I wanted to point it out explicitly.
Sorry for exaggerated phrasing.
As you implied, it’s enough to calibrate enough in an action-relevant way (e.g. “whether to follow the wager”), though I consider that to be more of a computational trick.
By the way, the way you phrased it in conclusions made it too similar to the fallacious “Abstractions are too weak for real phenomena” for my liking.