# Probability interpretations: Examples

(Writ­ten for Ar­bital in 2016.)

## Bet­ting on one-time events

Con­sider eval­u­at­ing, in June of 2016, the ques­tion: “What is the prob­a­bil­ity of Hillary Clin­ton win­ning the 2016 US pres­i­den­tial elec­tion?”

On the propen­sity view, Hillary has some fun­da­men­tal chance of win­ning the elec­tion. To ask about the prob­a­bil­ity is to ask about this ob­jec­tive chance. If we see a pre­dic­tion mar­ket in which prices move af­ter each new poll — so that it says 60% one day, and 80% a week later — then clearly the pre­dic­tion mar­ket isn’t giv­ing us very strong in­for­ma­tion about this ob­jec­tive chance, since it doesn’t seem very likely that Clin­ton’s real chance of win­ning is swing­ing so rapidly.

On the fre­quen­tist view, we can­not for­mally or rigor­ously say any­thing about the 2016 pres­i­den­tial elec­tion, be­cause it only hap­pens once. We can’t ob­serve a fre­quency with which Clin­ton wins pres­i­den­tial elec­tions. A fre­quen­tist might con­cede that they would cheer­fully buy for $1 a ticket that pays$20 if Clin­ton wins, con­sid­er­ing this a fa­vor­able bet in an in­for­mal sense, while in­sist­ing that this sort of rea­son­ing isn’t suffi­ciently rigor­ous, and there­fore isn’t suit­able for be­ing in­cluded in sci­ence jour­nals.

On the sub­jec­tive view, say­ing that Hillary has an 80% chance of win­ning the elec­tion sum­ma­rizes our knowl­edge about the elec­tion or our state of un­cer­tainty given what we cur­rently know. It makes sense for the pre­dic­tion mar­ket prices to change in re­sponse to new polls, be­cause our cur­rent state of knowl­edge is chang­ing.

## A coin with an un­known bias

Sup­pose we have a coin, weighted so that it lands heads some­where be­tween 0% and 100% of the time, but we don’t know the coin’s ac­tual bias.

The coin is then flipped three times where we can see it. It comes up heads twice, and tails once: HHT.

The coin is then flipped again, where no­body can see it yet. An hon­est and trust­wor­thy ex­per­i­menter lets you spin a wheel-of-gam­bling-odds — re­duc­ing the worry that the ex­per­i­menter might know more about the coin than you, and be offer­ing you a de­liber­ately rigged bet — and the wheel lands on (2 : 1). The ex­per­i­menter asks if you’d en­ter into a gam­ble where you win $2 if the un­seen coin flip is tails, and pay$1 if the un­seen coin flip is heads.

On a propen­sity view, the coin has some ob­jec­tive prob­a­bil­ity be­tween 0 and 1 of be­ing heads, but we just don’t know what this prob­a­bil­ity is. See­ing HHT tells us that the coin isn’t all-heads or all-tails, but we’re still just guess­ing — we don’t re­ally know the an­swer, and can’t say whether the bet is a fair bet.

On a fre­quen­tist view, the coin would (if flipped re­peat­edly) pro­duce some long-run fre­quency of heads that is be­tween 0 and 1. If we kept flip­ping the coin long enough, the ac­tual pro­por­tion of ob­served heads is guaran­teed to ap­proach ar­bi­trar­ily closely, even­tu­ally. We can’t say that the next coin flip is guaran­teed to be H or T, but we can make an ob­jec­tively true state­ment that will ap­proach to within ep­silon if we con­tinue to flip the coin long enough.

To de­cide whether or not to take the bet, a fre­quen­tist might try to ap­ply an un­bi­ased es­ti­ma­tor to the data we have so far. An “un­bi­ased es­ti­ma­tor” is a rule for tak­ing an ob­ser­va­tion and pro­duc­ing an es­ti­mate of , such that the ex­pected value of is . In other words, a fre­quen­tist wants a rule such that, if the hid­den bias of the coin was in fact to yield 75% heads, and we re­peat many times the op­er­a­tion of flip­ping the coin a few times and then ask­ing a new fre­quen­tist to es­ti­mate the coin’s bias us­ing this rule, the av­er­age value of the es­ti­mated bias will be 0.75. This is a prop­erty of the es­ti­ma­tion rule which is ob­jec­tive. We can’t hope for a rule that will always, in any par­tic­u­lar case, yield the true from just a few coin flips; but we can have a rule which will prov­ably have an av­er­age es­ti­mate of , if the ex­per­i­ment is re­peated many times.

In this case, a sim­ple un­bi­ased es­ti­ma­tor is to guess that the coin’s bias is equal to the ob­served pro­por­tion of heads, or 23. In other words, if we re­peat this ex­per­i­ment many many times, and when­ever we see heads in 3 tosses we guess that the coin’s bias is , then this rule definitely is an un­bi­ased es­ti­ma­tor. This es­ti­ma­tor says that a bet of $2 vs.$1 is fair, mean­ing that it doesn’t yield an ex­pected profit, so we have no rea­son to take the bet.

On a sub­jec­tivist view, we start out per­son­ally un­sure of where the bias lies within the in­ter­val [0, 1]. Un­less we have any knowl­edge or sus­pi­cion lead­ing us to think oth­er­wise, the coin is just as likely to have a bias be­tween 33% and 34%, as to have a bias be­tween 66% and 67%; there’s no rea­son to think it’s more likely to be in one range or the other.

Each coin flip we see is then ev­i­dence about the value of , since a flip H hap­pens with differ­ent prob­a­bil­ities de­pend­ing on the differ­ent val­ues of , and we up­date our be­liefs about us­ing Bayes’ rule. For ex­am­ple, H is twice as likely if than if so by Bayes’s Rule we should now think is twice as likely to lie near as it is to lie near .

When we start with a uniform prior, ob­serve mul­ti­ple flips of a coin with an un­known bias, see heads and tails, and then try to es­ti­mate the odds of the next flip com­ing up heads, the re­sult is Laplace’s Rule of Suc­ces­sion which es­ti­mates () : () for a prob­a­bil­ity of .

In this case, af­ter ob­serv­ing HHT, we es­ti­mate odds of 2 : 3 for tails vs. heads on the next flip. This makes a gam­ble that wins $2 on tails and loses$1 on heads a prof­itable gam­ble in ex­pec­ta­tion, so we take the bet.

Our choice of a uniform prior over was a lit­tle du­bi­ous — it’s the ob­vi­ous way to ex­press to­tal ig­no­rance about the bias of the coin, but ob­vi­ous­ness isn’t ev­ery­thing. (For ex­am­ple, maybe we ac­tu­ally be­lieve that a fair coin is more likely than a coin bi­ased 50.0000023% to­wards heads.) How­ever, all the rea­son­ing af­ter the choice of prior was rigor­ous ac­cord­ing to the laws of prob­a­bil­ity the­ory, which is the only method of ma­nipu­lat­ing quan­tified un­cer­tainty that obeys ob­vi­ous-seem­ing rules about how sub­jec­tive un­cer­tainty should be­have.

## Prob­a­bil­ity that the 98,765th dec­i­mal digit of π is 0

What is the prob­a­bil­ity that the 98,765th digit in the dec­i­mal ex­pan­sion of π is 0?

The propen­sity and fre­quen­tist views re­gard as non­sense the no­tion that we could talk about the prob­a­bil­ity of a math­e­mat­i­cal fact. Either the 98,765th dec­i­mal digit of π is or it’s not. If we’re run­ning re­peated ex­per­i­ments with a ran­dom num­ber gen­er­a­tor, and look­ing at differ­ent digits of π, then it might make sense to say that the ran­dom num­ber gen­er­a­tor has a 10% prob­a­bil­ity of pick­ing num­bers whose cor­re­spond­ing dec­i­mal digit of π is . But if we’re just pick­ing a non-ran­dom num­ber like 98,765, there’s no sense in which we could say that the 98,765th digit of π has a 10% propen­sity to be , or that this digit is with 10% fre­quency in the long run.

The sub­jec­tivist con­sid­ers prob­a­bil­ities to just re­fer to their own un­cer­tainty. So if a sub­jec­tivist has picked the num­ber 98,765 with­out yet know­ing the cor­re­spond­ing digit of π, and hasn’t made any ob­ser­va­tion that is known to them to be en­tan­gled with the 98,765th digit of π, and they’re pretty sure their friend hasn’t yet looked up the 98,765th digit of π ei­ther, and their friend offers a whim­si­cal gam­ble that costs $1 if the digit is non-zero and pays$20 if the digit is zero, the Bayesian takes the bet.

Note that this demon­strates a differ­ence be­tween the sub­jec­tivist in­ter­pre­ta­tion of “prob­a­bil­ity” and Bayesian prob­a­bil­ity the­ory. A perfect Bayesian rea­soner that knows the rules of logic and the defi­ni­tion of π must, by the ax­ioms of prob­a­bil­ity the­ory, as­sign prob­a­bil­ity ei­ther 0 or 1 to the claim “the 98,765th digit of π is a ” (de­pend­ing on whether or not it is). This is one of the rea­sons why perfect Bayesian rea­son­ing is in­tractable. A sub­jec­tivist that is not a perfect Bayesian nev­er­the­less claims that they are per­son­ally un­cer­tain about the value of the 98,765th digit of π. For­mal­iz­ing the rules of sub­jec­tive prob­a­bil­ities about math­e­mat­i­cal facts (in the way that prob­a­bil­ity the­ory for­mal­ized the rules for ma­nipu­lat­ing sub­jec­tive prob­a­bil­ities about em­piri­cal facts, such as which way a coin came up) is an open prob­lem; this in known as the prob­lem of log­i­cal un­cer­tainty.

• “The propen­sity and fre­quen­tist views re­gard as non­sense the no­tion that we could talk about the prob­a­bil­ity of a math­e­mat­i­cal fact”—couldn’t a fre­quen­tist define a refer­ence class us­ing all the digits of Pi? And then as­sume that the per­son knows noth­ing about Pi so that they throw away the place of the digit?

• A perfect Bayesian rea­soner that knows the rules of logic and the defi­ni­tion of π must, by the ax­ioms of prob­a­bil­ity the­ory, as­sign prob­a­bil­ity ei­ther 0 or 1 to the claim “the 98,765th digit of π is a 0” (de­pend­ing on whether or not it is). This is one of the rea­sons why perfect Bayesian rea­son­ing is in­tractable. A sub­jec­tivist that is not a perfect Bayesian nev­er­the­less claims that they are per­son­ally un­cer­tain about the value of the 98,765th digit of π.

The term “perfect Bayesian” sounds mis­lead­ing, there is noth­ing perfect about one’s in­abil­ity to make good prob­a­bil­ity es­ti­mates. This is like say­ing a “perfect two-boxer”.

On a re­lated note, what you call the open prob­lem of log­i­cal un­cer­tainty is one of the cases where pos­tu­lat­ing an ob­jec­tive re­al­ity (in this case, a math­e­mat­i­cal re­al­ity), also known on this site as “the ter­ri­tory” runs into limi­ta­tions. Once you stop in­sist­ing that any yet un­mea­sured value or an un­proven the­o­rem is ei­ther true or false (or un­de­cid­able), but go with the more in­tu­ition­ist ap­proach, the made-up con­tra­dic­tion be­tween “but there is a 98,765th digit of π out there that has a definite value” and “be­fore calcu­lat­ing the 8,765th digit of π (in effect, mak­ing an ob­ser­va­tion) the best model of π pre­dicts equal prob­a­bil­ity of all digits” dis­solves.

• I think I un­der­stand what your view means with re­spect to phys­i­cal un­cer­tainty, but I’m not sure what it means w.r.t. log­i­cal un­cer­tainty. Surely, there must be some fact of the mat­ter about what the ra­tio of a cir­cle’s cir­cum­fer­ence to its di­ame­ter is? Or is there not? And if there is, does that not im­ply some fact of the mat­ter about any given digit of π, even if I don’t know what said digit is?

• Surely, there must be some fact of the mat­ter about what the ra­tio of a cir­cle’s cir­cum­fer­ence to its di­ame­ter is?

This is ex­actly the is­sue at hand. You be­lieve in ex­ter­nal math­e­mat­i­cal “facts”, ideal pla­tonic ob­jects. The math­e­mat­i­cal ter­ri­tory. This is a use­ful be­lief at times, but not in this case, as it gets in the way of mak­ing oth­er­wise ob­vi­ous pre­dic­tions about ob­ser­va­tions, such as “how likely that a ran­domly picked digit of π is zero, once it is picked, but not yet calcu­lated?”

• Well, let me put it an­other way. Sup­pose that I calcu­late the 98,765th digit of π. And my friend Hasan, who lives on the other side of the world, also, sep­a­rately, calcu­lates the 98,765th digit of π. Can we get differ­ent re­sults? (Other than by mak­ing some mis­take in writ­ing the code that does the calcu­la­tion, or some such.) Is that a thing that can hap­pen? What is the prob­a­bil­ity of the 98,765th digit of π be­ing one thing when calcu­lated by one per­son, but some­thing else when calcu­lated by some­one else, el­se­where? (And if nonzero, how far does this go—could the 1,500th digit of π vary from per­son to per­son? The 220th? The 30th? The 3rd?!)

If you say that this sort of thing can hap­pen, well, then you’re cer­tainly say­ing some­thing novel and strange. I guess all I have to say to that is “[cita­tion needed]”. But, if (as seems more likely) you agree that such a thing can­not hap­pen, then my ques­tion is: just what ex­actly is it that makes the 98,765th of π be the same thing when calcu­lated by me, or by Hasan, or by any­one else? What­ever that thing is, what is wrong with call­ing it “a fact of the mat­ter about what the 98,765th digit of π is”?

• You seem to be con­flat­ing two differ­ent ques­tions:

What is your best es­ti­mate of prob­a­bil­ity of the cur­rently un­known to you 98,765th digit of π com­ing out zero, once some­one calcu­lates it?

and

What is your best es­ti­mate of prob­a­bil­ity of the 98,765th digit of π calcu­lated by two differ­ent peo­ple be­ing differ­ent?

Once enough peo­ple re­li­ably do the same calcu­la­tion (or if there is an­other re­li­able way to perform the ob­ser­va­tion of the 98,765th digit of π), then it can be added to the list of performed ob­ser­va­tions and, if needed used to pre­dict fu­ture ob­ser­va­tions.

just what ex­actly is it that makes the 98,765th of π be the same thing when calcu­lated by me, or by Hasan, or by any­one else? What­ever that thing is, what is wrong with call­ing it “a fact of the mat­ter about what the 98,765th digit of π is”

This goes back to re­al­ism vs anti-re­al­ism, not any­thing I had in­vented. Anti-re­al­ism is a self-con­sis­tent episte­mol­ogy, it pops up in many ar­eas in­de­pen­dently. Ac­cord­ing to Wikipe­dia, in sci­ence an ex­am­ple of it in sci­ence is in­stru­men­tal­ism, and in math it is in­tu­ition­ism: “there are no non-ex­pe­rienced math­e­mat­i­cal truths”.

There is no differ­ence be­tween log­i­cal un­cer­tainty and en­vi­ron­men­tal un­cer­tainty in anti-re­al­ism. OP seems to have rein­vented the jux­ta­po­si­tion of re­al­ism and anti-re­al­ism in the set­ting of the prob­a­bil­ity the­ory, call­ing it “perfect Bayesi­anism” and “sub­jec­tive Bayesi­anism” re­spec­tively. And “perfect Bayesi­anism” runs into trou­ble with log­i­cal vs en­vi­ron­men­tal un­cer­tain­ties, be­cause of the ex­tra (and un­nec­es­sary, in the anti-re­al­ist view) pos­tu­late of ob­jec­tive re­al­ity.

• I still don’t think you’ve an­swered Said’s ques­tion. The ques­tion is whether two peo­ple can ob­serve differ­ent val­ues of pi. Or, to put it differ­ently, why is it that, when­ever any­one com­putes a value of pi, it seems to come out to the same value (3.14159...). Doesn’t that in­di­cate that there is some kind of ob­jec­tive re­al­ity, to which our math­e­mat­ics cor­re­sponds?

One of the ques­tions that Wigner brings up in The Un­rea­son­able Effec­tive­ness of Math­e­mat­ics in the Nat­u­ral Sciences is why does our math work so well at pre­dict­ing the fu­ture? I would put the same ques­tion to you, but in a more gen­eral form. If there is no such thing as non-ex­pe­rienced math­e­mat­i­cal truths, then why does ev­ery­one’s ex­pe­rience of math­e­mat­i­cal truths seem to be the same?

• Doesn’t that in­di­cate that there is some kind of ob­jec­tive re­al­ity, to which our math­e­mat­ics cor­re­sponds?

A re­al­ity be­hind re­peat­able ob­ser­va­tions is a good model, as long as it works. My point is that it doesn’t always work, like in the con­fu­sion about log­i­cal un­cer­tainty.

And I dis­agree with the as­sump­tions be­hind the Wigner’s ques­tion, “why does our math work so well at pre­dict­ing the fu­ture?”, speci­fi­cally that math’s effec­tive­ness is “un­rea­son­able”. Hu­man and an­i­mal brains do com­pli­cated calcu­la­tions all the time in real time to get through life, like solv­ing what amounts to non-lin­ear par­tial differ­en­tial equa­tions to even get a bite of food into your mouth. Just be­cause it is sub­con­scious, it is no less of a math than prov­ing the­o­rems. What most hu­mans mean by math is con­struct­ing con­scious, not sub­con­scious meta-mod­els and us­ing them in mul­ti­ple con­texts. But we sub­con­scious meta-mod­el­ing like this all the time in other ar­eas of hu­man ex­pe­rience, so my an­swer to Wigner’s ques­tion is “you are com­mit­ting a mind pro­jec­tion fal­lacy, the ap­par­ently un­rea­son­able effec­tive­ness of math­e­mat­ics is a state­ment about hu­man mind, not about the world”.

If there is no such thing as non-ex­pe­rienced math­e­mat­i­cal truths, then why does ev­ery­one’s ex­pe­rience of math­e­mat­i­cal truths seem to be the same?

In gen­eral, how­ever, your ques­tions about the in­tu­ition­ist ap­proach to math is best di­rected to pro­fes­sional math­e­mat­i­ci­ans who are ac­tu­ally in­tu­ition­ists, though.

• Hu­man and an­i­mal brains do com­pli­cated calcu­la­tions all the time in real time to get through life, like solv­ing what amounts to non-lin­ear par­tial differ­en­tial equa­tions to even get a bite of food into your mouth. Just be­cause it is sub­con­scious, it is no less of a math than prov­ing the­o­rems.

I agree. So if there is no “ob­jec­tive” re­al­ity, apart from that which we ex­pe­rience, then why is it that we all seem to ex­pe­rience the same re­al­ity? When I shoot a bas­ket­ball, or hit a ten­nis ball, both I and the referee see the same tra­jec­tory and are in ap­prox­i­mate agree­ment about where the ball lands. When I lift a piece of food to my mouth and eat it, it would sur­prise me if some­one across the table said that they saw it spill from my fork and stain my shirt.

In the ab­sence of an ex­ter­nal re­al­ity, why is it that ev­ery­one’s model of the world ap­pears to be in such con­cor­dance with ev­ery­one else’s?

• So if there is no “ob­jec­tive” re­al­ity, apart from that which we ex­pe­rience, then why is it that we all seem to ex­pe­rience the same re­al­ity?

I am not say­ing that there is no ob­jec­tive re­al­ity, just that I am ag­nos­tic about it. In the ex­am­ple you de­scribe, it is a use­ful meta-model, though not all the time. You may no­tice that, de­spite a video re­view and slow mo­tion hi-res cam­eras, fans of differ­ent teams still ar­gue about what hap­pened, and the fi­nal de­ci­sion is in the hands of a referee. You and your part­ner (es­pe­cially ex part­ner) may dis­agree about “what re­ally hap­pened” and there is of­ten no way to tell “who is right”. One in­stead has to ac­cept that what one per­son ex­pe­rienced is not nec­es­sar­ily what an­other did, and, at least in­stru­men­tally, ar­gu­ing about whose re­al­ity is the “true” is likely to be not use­ful at all. One may as well ac­cept the model where some­what differ­ent things hap­pened to differ­ent ac­tors.

In the ab­sence of an ex­ter­nal re­al­ity, why is it that ev­ery­one’s model of the world ap­pears to be in such con­cor­dance with ev­ery­one else’s?

Does it? Who won the World War II, Amer­i­cans, Bri­tish or Rus­si­ans? Is Trump a hero or a villain? Did Elon Musk dis­close ma­te­rial in­for­ma­tion or not in his tweets? Do math­e­mat­i­cal in­fini­ties ex­ist? Are the laws of physics in­vented or dis­cov­ered? Was Je­sus a son of God? The list of dis­agree­ments about “ob­jec­tive re­al­ity” is end­less. Sure, there is some “con­cor­dance” be­tween differ­ent peo­ple’s views of the world, but it is much less strong than one naively as­sumes.

• The ex­am­ples you use re­in­force my point. We ar­gue about ex­tremely fine de­tails. When sup­port­ers of op­pos­ing teams ar­gue over whether a point was or was not scored, they’re dis­put­ing whether the ball was here or there by a few mil­lime­ters. You won’t find very many peo­ple ar­gu­ing that ac­tu­ally, the ball was clear on the other side of the field and in re­al­ity, the dis­puted point is one that would have been scored by the other team.

Similarly, we might ar­gue about whether the Bri­tish, Amer­i­cans or Rus­si­ans were pri­mar­ily re­spon­si­ble for the United Na­tions’ vic­tory in World War 2, but I don’t think you’ll find very many peo­ple ar­gu­ing that ac­tu­ally it was the Ital­i­ans who won World War 2.

The fact that our per­cep­tions of re­al­ity match each other 99.999% of the time, to me, in­di­cates that there’s some­thing out there that ex­ists re­gard­less of whether I per­ceive it or not. I call that “re­al­ity”.

• I can see your point, and it’s the one most peo­ple im­plic­itly ac­cept. Ob­ser­va­tions are pre­dictable, there­fore there is a shared re­al­ity out there gen­er­at­ing those ob­ser­va­tions. It works most of the time. But in the edge cases (or “ex­tremely fine de­tails”) this im­plicit as­sump­tion breaks down. Like in the case of “ob­jec­tive math­e­mat­i­cal facts wait­ing to be dis­cov­ered”, such as the 98,765th of π be­fore you mea­sure it. So why in­sist on ap­ply­ing this as­sump­tion out­side of its realm of ap­pli­ca­bil­ity? Isn’t it sort of like in­sist­ing that if you shoot a bul­let from a ship mov­ing with nearly the speed of light, it will travel faster than light?

• You seem to be say­ing that “ex­ter­nal shared re­al­ity” is an ap­prox­i­ma­tion in the same way that New­to­nian me­chan­ics is an ap­prox­i­ma­tion for Ein­stei­nian rel­a­tivity. That’s fine. So what is “ex­ter­nal shared re­al­ity” an ap­prox­i­ma­tion of? Just what ex­actly is out there gen­er­at­ing in­puts to my senses, and by what mechanism does it re­main in sync with ev­ery­one else (ap­prox­i­mately)?

• Just what ex­actly is out there gen­er­at­ing in­puts to my senses, and by what mechanism does it re­main in sync with ev­ery­one else (ap­prox­i­mately)?

Some­times the “out there” can be mod­eled as a shared re­al­ity, sure. The key word is “mod­eled”. Some­times this model is not a good one. If you in­sist on priv­ileg­ing one model over all oth­ers to be the true ob­jec­tive ex­ter­nal re­al­ity valid ev­ery­where, you pay the price where it fails. Like in the OP’s case.