Probability is Subjectively Objective

Fol­lowup to: Prob­a­bil­ity is in the Mind

“Real­ity is that which, when you stop be­liev­ing in it, doesn’t go away.”
—Philip K. Dick

There are two kinds of Bayesi­ans, allegedly. Sub­jec­tive Bayesi­ans be­lieve that “prob­a­bil­ities” are de­grees of un­cer­tainty ex­ist­ing in our minds; if you are un­cer­tain about a phe­nomenon, that is a fact about your state of mind, not a prop­erty of the phe­nomenon it­self; prob­a­bil­ity the­ory con­strains the log­i­cal co­her­ence of un­cer­tain be­liefs. Then there are ob­jec­tive Bayesi­ans, who… I’m not quite sure what it means to be an “ob­jec­tive Bayesian”; there are mul­ti­ple defi­ni­tions out there. As best I can tell, an “ob­jec­tive Bayesian” is any­one who uses Bayesian meth­ods and isn’t a sub­jec­tive Bayesian.

If I re­call cor­rectly, E. T. Jaynes, mas­ter of the art, once de­scribed him­self as a sub­jec­tive-ob­jec­tive Bayesian. Jaynes cer­tainly be­lieved very firmly that prob­a­bil­ity was in the mind; Jaynes was the one who coined the term Mind Pro­jec­tion Fal­lacy. But Jaynes also didn’t think that this im­plied a li­cense to make up what­ever pri­ors you liked. There was only one cor­rect prior dis­tri­bu­tion to use, given your state of par­tial in­for­ma­tion at the start of the prob­lem.

How can some­thing be in the mind, yet still be ob­jec­tive?

It ap­pears to me that a good deal of philo­soph­i­cal ma­tu­rity con­sists in be­ing able to keep sep­a­rate track of nearby con­cepts, with­out mix­ing them up.

For ex­am­ple, to un­der­stand evolu­tion­ary psy­chol­ogy, you have to keep sep­a­rate track of the psy­cholog­i­cal pur­pose of an act, and the evolu­tion­ary pseudo-pur­poses of the adap­ta­tions that ex­e­cute as the psy­chol­ogy; this is a com­mon failure of new­com­ers to evolu­tion­ary psy­chol­ogy, who read, mi­s­un­der­stand, and there­after say, “You think you love your chil­dren, but you’re just try­ing to max­i­mize your fit­ness!”

What is it, ex­actly, that the terms “sub­jec­tive” and “ob­jec­tive”, mean? Let’s say that I hand you a sock. Is it a sub­jec­tive or an ob­jec­tive sock? You be­lieve that 2 + 3 = 5. Is your be­lief sub­jec­tive or ob­jec­tive? What about two plus three ac­tu­ally equal­ing five—is that sub­jec­tive or ob­jec­tive? What about a spe­cific act of adding two ap­ples and three ap­ples and get­ting five ap­ples?

I don’t in­tend to con­fuse you in shrouds of words; but I do mean to point out that, while you may feel that you know very well what is “sub­jec­tive” or “ob­jec­tive”, you might find that you have a bit of trou­ble say­ing out loud what those words mean.

Sup­pose there’s a calcu­la­tor that com­putes “2 + 3 = 5”. We punch in “2“, then “+”, then “3”, and lo and be­hold, we see “5” flash on the screen. We ac­cept this as ev­i­dence that 2 + 3 = 5, but we wouldn’t say that the calcu­la­tor’s phys­i­cal out­put defines the an­swer to the ques­tion 2 + 3 = ?. A cos­mic ray could strike a tran­sis­tor, which might give us mis­lead­ing ev­i­dence and cause us to be­lieve that 2 + 3 = 6, but it wouldn’t af­fect the ac­tual sum of 2 + 3.

Which propo­si­tion is com­mon-sen­si­cally true, but philo­soph­i­cally in­ter­est­ing: while we can eas­ily point to the phys­i­cal lo­ca­tion of a sym­bol on a calcu­la­tor screen, or ob­serve the re­sult of putting two ap­ples on a table fol­lowed by an­other three ap­ples, it is rather harder to track down the where­abouts of 2 + 3 = 5. (Did you look in the garage?)

But let us leave aside the ques­tion of where the fact 2 + 3 = 5 is lo­cated—in the uni­verse, or some­where else—and con­sider the as­ser­tion that the propo­si­tion is “ob­jec­tive”. If a cos­mic ray strikes a calcu­la­tor and makes it out­put “6“ in re­sponse to the query “2 + 3 = ?”, and you add two ap­ples to a table fol­lowed by three ap­ples, then you’ll still see five ap­ples on the table. If you do the calcu­la­tion in your own head, ex­pend­ing the nec­es­sary com­put­ing power—we as­sume that 2 + 3 is a very difficult sum to com­pute, so that the an­swer is not im­me­di­ately ob­vi­ous to you—then you’ll get the an­swer “5”. So the cos­mic ray strike didn’t change any­thing.

And similarly—ex­actly similarly—what if a cos­mic ray strikes a neu­ron in­side your brain, caus­ing you to com­pute “2 + 3 = 7”? Then, adding two ap­ples to three ap­ples, you will ex­pect to see seven ap­ples, but in­stead you will be sur­prised to see five ap­ples.

If in­stead we found that no one was ever mis­taken about ad­di­tion prob­lems, and that, more­over, you could change the an­swer by an act of will, then we might be tempted to call ad­di­tion “sub­jec­tive” rather than “ob­jec­tive”. I am not say­ing that this is ev­ery­thing peo­ple mean by “sub­jec­tive” and “ob­jec­tive”, just point­ing to one as­pect of the con­cept. One might sum­ma­rize this as­pect thus: “If you can change some­thing by think­ing differ­ently, it’s sub­jec­tive; if you can’t change it by any­thing you do strictly in­side your head, it’s ob­jec­tive.”

Mind is not magic. Every act of rea­son­ing that we hu­man be­ings carry out, is com­puted within some par­tic­u­lar hu­man brain. But not ev­ery com­pu­ta­tion is about the state of a hu­man brain. Not ev­ery thought that you think is about some­thing that can be changed by think­ing. Herein lies the op­por­tu­nity for con­fu­sion-of-lev­els. The quo­ta­tion is not the refer­ent. If you are go­ing to con­sider thoughts as refer­en­tial at all—if not, I’d like you to ex­plain the mys­te­ri­ous cor­re­la­tion be­tween my thought “2 + 3 = 5” and the ob­served be­hav­ior of ap­ples on ta­bles—then, while the quoted thoughts will always change with thoughts, the refer­ents may or may not be en­tities that change with chang­ing hu­man thoughts.

The calcu­la­tor com­putes “What is 2 + 3?”, not “What does this calcu­la­tor com­pute as the re­sult of 2 + 3?” The an­swer to the former ques­tion is 5, but if the calcu­la­tor were to ask the lat­ter ques­tion in­stead, the re­sult could self-con­sis­tently be any­thing at all! If the calcu­la­tor re­turned 42, then in­deed, “What does this calcu­la­tor com­pute as the re­sult of 2 + 3?” would in fact be 42.

So just be­cause a com­pu­ta­tion takes place in­side your brain, does not mean that the com­pu­ta­tion ex­plic­itly men­tions your brain, that it has your brain as a refer­ent, any more than the calcu­la­tor men­tions the calcu­la­tor. The calcu­la­tor does not at­tempt to con­tain a rep­re­sen­ta­tion of it­self, only of num­bers.

In­deed, in the most straight­for­ward im­ple­men­ta­tion, the calcu­la­tor that asks “What does this calcu­la­tor com­pute as the an­swer to the query 2 + 3 = ?” will never re­turn a re­sult, just simu­late it­self simu­lat­ing it­self un­til it runs out of mem­ory.

But if you punch the keys “2”, “+”, and “3”, and the calcu­la­tor pro­ceeds to com­pute “What do I out­put when some­one punches ‘2 + 3’?”, the re­sult­ing com­pu­ta­tion does have one in­ter­est­ing char­ac­ter­is­tic: the refer­ent of the com­pu­ta­tion is highly sub­jec­tive, since it de­pends on the com­pu­ta­tion, and can be made to be any­thing just by chang­ing the com­pu­ta­tion.

Is prob­a­bil­ity, then, sub­jec­tive or ob­jec­tive?

Well, prob­a­bil­ity is com­puted within hu­man brains or other calcu­la­tors. A prob­a­bil­ity is a state of par­tial in­for­ma­tion that is pos­sessed by you; if you flip a coin and press it to your arm, the coin is show­ing heads or tails, but you as­sign the prob­a­bil­ity 12 un­til you re­veal it. A friend, who got a tiny but not fully in­for­ma­tive peek, might as­sign a prob­a­bil­ity of 0.6.

So can you make the prob­a­bil­ity of win­ning the lot­tery be any­thing you like?

For­get about many-wor­lds for the mo­ment—you should al­most always be able to for­get about many-wor­lds—and pre­tend that you’re liv­ing in a sin­gle Small World where the lot­tery has only a sin­gle out­come. You will nonethe­less have a need to call upon prob­a­bil­ity. Or if you pre­fer, we can dis­cuss the ten trillionth dec­i­mal digit of pi, which I be­lieve is not yet known. (If you are fool­ish enough to re­fuse to as­sign a prob­a­bil­ity dis­tri­bu­tion to this en­tity, you might pass up an ex­cel­lent bet, like bet­ting $1 to win $1000 that the digit is not 4.) Your un­cer­tainty is a state of your mind, of par­tial in­for­ma­tion that you pos­sess. Some­one else might have differ­ent in­for­ma­tion, com­plete or par­tial. And the en­tity it­self will only ever take on a sin­gle value.

So can you make the prob­a­bil­ity of win­ning the lot­tery, or the prob­a­bil­ity of the ten trillionth dec­i­mal digit of pi equal­ing 4, be any­thing you like?

You might be tempted to re­ply: “Well, since I cur­rently think the prob­a­bil­ity of win­ning the lot­tery is one in a hun­dred mil­lion, then ob­vi­ously, I will cur­rently ex­pect that as­sign­ing any other prob­a­bil­ity than this to the lot­tery, will de­crease my ex­pected log-score—or if you pre­fer a de­ci­sion-the­o­retic for­mu­la­tion, I will ex­pect this mod­ifi­ca­tion to my­self to de­crease ex­pected util­ity. So, ob­vi­ously, I will not choose to mod­ify my prob­a­bil­ity dis­tri­bu­tion. It wouldn’t be re­flec­tively co­her­ent.”

So re­flec­tive co­herency is the goal, is it? Too bad you weren’t born with a prior that as­signed prob­a­bil­ity 0.9 to win­ning the lot­tery! Then, by ex­actly the same line of ar­gu­ment, you wouldn’t want to as­sign any prob­a­bil­ity ex­cept 0.9 to win­ning the lot­tery. And you would still be re­flec­tively co­her­ent. And you would have a 90% prob­a­bil­ity of win­ning mil­lions of dol­lars! Hooray!

“No, then I would think I had a 90% prob­a­bil­ity of win­ning the lot­tery, but ac­tu­ally, the prob­a­bil­ity would only be one in a hun­dred mil­lion.”

Well, of course you would be ex­pected to say that. And if you’d been born with a prior that as­signed 90% prob­a­bil­ity to your win­ning the lot­tery, you’d con­sider an alleged prob­a­bil­ity of 10^-8, and say, “No, then I would think I had al­most no prob­a­bil­ity of win­ning the lot­tery, but ac­tu­ally, the prob­a­bil­ity would be 0.9.”

“Yeah? Then just mod­ify your prob­a­bil­ity dis­tri­bu­tion, and buy a lot­tery ticket, and then wait and see what hap­pens.”

What hap­pens? Either the ticket will win, or it won’t. That’s what will hap­pen. We won’t get to see that some par­tic­u­lar prob­a­bil­ity was, in fact, the ex­actly right prob­a­bil­ity to as­sign.

“Perform the ex­per­i­ment a hun­dred times, and—”

Okay, let’s talk about the ten trillionth digit of pi, then. Sin­gle-shot prob­lem, no “long run” you can mea­sure.

Prob­a­bil­ity is sub­jec­tively ob­jec­tive: Prob­a­bil­ity ex­ists in your mind: if you’re ig­no­rant of a phe­nomenon, that’s an at­tribute of you, not an at­tribute of the phe­nomenon. Yet it will seem to you that you can’t change prob­a­bil­ities by wish­ing.

You could make your­self com­pute some­thing else, per­haps, rather than prob­a­bil­ity. You could com­pute “What do I say is the prob­a­bil­ity?” (an­swer: any­thing you say) or “What do I wish were the prob­a­bil­ity?” (an­swer: what­ever you wish) but these things are not the prob­a­bil­ity, which is sub­jec­tively ob­jec­tive.

The thing about sub­jec­tively ob­jec­tive quan­tities is that they re­ally do seem ob­jec­tive to you. You don’t look them over and say, “Oh, well, of course I don’t want to mod­ify my own prob­a­bil­ity es­ti­mate, be­cause no one can just mod­ify their prob­a­bil­ity es­ti­mate; but if I’d been born with a differ­ent prior I’d be say­ing some­thing differ­ent, and I wouldn’t want to mod­ify that ei­ther; and so none of us is su­pe­rior to any­one else.” That’s the way a sub­jec­tively sub­jec­tive quan­tity would seem.

No, it will seem to you that, if the lot­tery sells a hun­dred mil­lion tick­ets, and you don’t get a peek at the re­sults, then the prob­a­bil­ity of a ticket win­ning, is one in a hun­dred mil­lion. And that you could be born with differ­ent pri­ors but that wouldn’t give you any bet­ter odds. And if there’s some­one next to you say­ing the same thing about their 90% prob­a­bil­ity es­ti­mate, you’ll just shrug and say, “Good luck with that.” You won’t ex­pect them to win.

Prob­a­bil­ity is sub­jec­tively re­ally ob­jec­tive, not just sub­jec­tively sort of ob­jec­tive.

Jaynes used to recom­mend that no one ever write out an un­con­di­tional prob­a­bil­ity: That you never, ever write sim­ply P(A), but always write P(A|I), where I is your prior in­for­ma­tion. I’ll use Q in­stead of I, for ease of read­ing, but Jaynes used I. Similarly, one would not write P(A|B) for the pos­te­rior prob­a­bil­ity of A given that we learn B, but rather P(A|B,Q), the prob­a­bil­ity of A given that we learn B and had back­ground in­for­ma­tion Q.

This is good ad­vice in a purely prag­matic sense, when you see how many false “para­doxes” are gen­er­ated by ac­ci­den­tally us­ing differ­ent prior in­for­ma­tion in differ­ent places.

But it also makes a deep philo­soph­i­cal point as well, which I never saw Jaynes spell out ex­plic­itly, but I think he would have ap­proved: there is no such thing as a prob­a­bil­ity that isn’t in any mind. Any mind that takes in ev­i­dence and out­puts prob­a­bil­ity es­ti­mates of the next event, re­mem­ber, can be viewed as a prior—so there is no prob­a­bil­ity with­out pri­ors/​minds.

You can’t un­wind the Q. You can’t ask “What is the un­con­di­tional prob­a­bil­ity of our back­ground in­for­ma­tion be­ing true, P(Q)?” To make that es­ti­mate, you would still need some kind of prior. No way to un­wind back to an ideal ghost of perfect empti­ness...

You might ar­gue that you and the lot­tery-ticket buyer do not re­ally have a dis­agree­ment about prob­a­bil­ity. You say that the prob­a­bil­ity of the ticket win­ning the lot­tery is one in a hun­dred mil­lion given your prior, P(W|Q1) = 10^-8. The other fel­low says the prob­a­bil­ity of the ticket win­ning given his prior is P(W|Q2) = 0.9. Every time you say “The prob­a­bil­ity of X is Y”, you re­ally mean, “P(X|Q1) = Y”. And when he says, “No, the prob­a­bil­ity of X is Z”, he re­ally means, “P(X|Q2) = Z”.

Now you might, if you traced out his math­e­mat­i­cal calcu­la­tions, agree that, in­deed, the con­di­tional prob­a­bil­ity of the ticket win­ning, given his weird prior is 0.9. But you wouldn’t agree that “the prob­a­bil­ity of the ticket win­ning” is 0.9. Just as he wouldn’t agree that “the prob­a­bil­ity of the ticket win­ning” is 10^-8.

Even if the two of you re­fer to differ­ent math­e­mat­i­cal calcu­la­tions when you say the word “prob­a­bil­ity”, you don’t think that puts you on equal ground, nei­ther of you be­ing bet­ter than the other. And nei­ther does he, of course.

So you see that, sub­jec­tively, prob­a­bil­ity re­ally does feel ob­jec­tive—even af­ter you have sub­jec­tively taken all ap­par­ent sub­jec­tivity into ac­count.

And this is not mis­taken, be­cause, by golly, the prob­a­bil­ity of win­ning the lot­tery re­ally is 10^-8, not 0.9. It’s not as if you’re do­ing your prob­a­bil­ity calcu­la­tion wrong, af­ter all. If you weren’t wor­ried about be­ing fair or about jus­tify­ing your­self to philoso­phers, if you only wanted to get the cor­rect an­swer, your bet­ting odds would be 10^-8.

Some­where out in mind de­sign space, there’s a mind with any pos­si­ble prior; but that doesn’t mean that you’ll say, “All pri­ors are cre­ated equal.”

When you judge those al­ter­nate minds, you’ll do so us­ing your own mind—your own be­liefs about the uni­verse—your own pos­te­rior that came out of your own prior, your own pos­te­rior prob­a­bil­ity as­sign­ments P(X|A,B,C,...,Q1). But there’s noth­ing wrong with that. It’s not like you could judge us­ing some­thing other than your­self. It’s not like you could have a prob­a­bil­ity as­sign­ment with­out any prior, a de­gree of un­cer­tainty that isn’t in any mind.

And so, when all that is said and done, it still seems like the prob­a­bil­ity of win­ning the lot­tery re­ally is 10^-8, not 0.9. No mat­ter what other minds in de­sign space say differ­ently.

Which shouldn’t be sur­pris­ing. When you com­pute prob­a­bil­ities, you’re think­ing about lot­tery balls, not think­ing about brains or mind de­signs or other peo­ple with differ­ent pri­ors. Your prob­a­bil­ity com­pu­ta­tion makes no men­tion of that, any more than it ex­plic­itly rep­re­sents it­self. Your goal, af­ter all, is to win, not to be fair. So of course prob­a­bil­ity will seem to be in­de­pen­dent of what other minds might think of it.

Okay, but… you still can’t win the lot­tery by as­sign­ing a higher prob­a­bil­ity to win­ning.

If you like, we could re­gard prob­a­bil­ity as an ideal­ized com­pu­ta­tion, just like 2 + 2 = 4 seems to be in­de­pen­dent of any par­tic­u­lar er­ror-prone calcu­la­tor that com­putes it; and you could re­gard your mind as try­ing to ap­prox­i­mate this ideal com­pu­ta­tion. In which case, it is good that your mind does not men­tion peo­ple’s opinions, and only thinks of the lot­tery balls; the ideal com­pu­ta­tion makes no men­tion of peo­ple’s opinions, and we are try­ing to re­flect this ideal as ac­cu­rately as pos­si­ble...

But what you will calcu­late is the “ideal calcu­la­tion” to plug into your bet­ting odds, will de­pend on your prior, even though the calcu­la­tion won’t have an ex­plicit de­pen­dency on “your prior”. Some­one who thought the uni­verse was anti-Oc­camian, would ad­vo­cate an anti-Oc­camian calcu­la­tion, re­gard­less of whether or not any­one thought the uni­verse was anti-Oc­camian.

Your calcu­la­tions get checked against re­al­ity, in a prob­a­bil­is­tic way; you ei­ther win the lot­tery or not. But in­ter­pret­ing these re­sults, is done with your prior; once again there is no prob­a­bil­ity that isn’t in any mind.

I am not try­ing to ar­gue that you can win the lot­tery by wish­ing, of course. Rather, I am try­ing to in­cul­cate the abil­ity to dis­t­in­guish be­tween lev­els.

When you think about the on­tolog­i­cal na­ture of prob­a­bil­ity, and perform re­duc­tion­ism on it—when you try to ex­plain how “prob­a­bil­ity” fits into a uni­verse in which states of mind do not ex­ist fun­da­men­tally—then you find that prob­a­bil­ity is com­puted within a brain; and you find that other pos­si­ble minds could perform mostly-analo­gous op­er­a­tions with differ­ent pri­ors and ar­rive at differ­ent an­swers.

But, when you con­sider prob­a­bil­ity as prob­a­bil­ity, think about the refer­ent in­stead of the thought pro­cess—which think­ing you will do in your own thoughts, which are phys­i­cal pro­cesses—then you will con­clude that the vast ma­jor­ity of pos­si­ble pri­ors are prob­a­bly wrong. (You will also be able to con­ceive of pri­ors which are, in fact, bet­ter than yours, be­cause they as­sign more prob­a­bil­ity to the ac­tual out­come; you just won’t know in ad­vance which al­ter­na­tive prior is the truly bet­ter one.)

If you again swap your gog­gles to think about how prob­a­bil­ity is im­ple­mented in the brain, the seem­ing ob­jec­tivity of prob­a­bil­ity is the way the prob­a­bil­ity al­gorithm feels from in­side; so it’s no mys­tery that, con­sid­er­ing prob­a­bil­ity as prob­a­bil­ity, you feel that it’s not sub­ject to your whims. That’s just what the prob­a­bil­ity-com­pu­ta­tion would be ex­pected to say, since the com­pu­ta­tion doesn’t rep­re­sent any de­pen­dency on your whims.

But when you swap out those gog­gles and go back to think­ing about prob­a­bil­ities, then, by golly, your al­gorithm seems to be right in com­put­ing that prob­a­bil­ity is not sub­ject to your whims. You can’t win the lot­tery just by chang­ing your be­liefs about it. And if that is the way you would be ex­pected to feel, then so what? The feel­ing has been ex­plained, not ex­plained away; it is not a mere feel­ing. Just be­cause a calcu­la­tion is im­ple­mented in your brain, doesn’t mean it’s wrong, af­ter all.

Your “prob­a­bil­ity that the ten trillionth dec­i­mal digit of pi is 4”, is an at­tribute of your­self, and ex­ists in your mind; the real digit is ei­ther 4 or not. And if you could change your be­lief about the prob­a­bil­ity by edit­ing your brain, you wouldn’t ex­pect that to change the prob­a­bil­ity.

There­fore I say of prob­a­bil­ity that it is “sub­jec­tively ob­jec­tive”.

Part of The Me­taethics Sequence

Next post: “Whither Mo­ral Progress?

Pre­vi­ous post: “Re­bel­ling Within Na­ture