# [Link] The Bayesian argument against induction.

In 1983 Karl Pop­per and David Miller pub­lished an ar­gu­ment to the effect that prob­a­bil­ity the­ory could be used to dis­prove in­duc­tion. Pop­per had long been an op­po­nent of in­duc­tion. Since prob­a­bil­ity the­ory in gen­eral, and Bayes in par­tic­u­lar is of­ten seen as res­cu­ing in­duc­tion from the stan­dard ob­jec­tions, the ar­gu­ment is sig­nifi­cant.

It is be­ing dis­cussed over at the Crit­i­cal Ra­tion­al­ism site.

• In­ter­est­ing but ex­tremely un­per­sua­sive.

I agreed with ev­ery­thing up un­til this point:

That is, to the ex­tent that B in­creases the prob­a­bil­ity of A, it does so by in­creas­ing the prob­a­bil­ity of A v B more than it de­creases the prob­a­bil­ity of A v ~B. How­ever, since A v B is a log­i­cal con­se­quence of B to be­gin with, the in­crease in prob­a­bil­ity is a purely de­duc­tive in­fer­ence.

This seems to be wordgames. Say­ing this is a de­duc­tive in­fer­ence misses the whole point that this an in­fer­ence which can only be used af­ter B has been ob­served. Other­wise it is just math.

The next para­graph seems to be similarly flawed.

The in­duc­tive view of prob­a­bil­is­tic in­fer­ence rests on the fal­lacy of de­com­po­si­tion, i.e. as­sum­ing that what is true for the whole must be true for its parts. Not only do log­i­cal con­se­quences of A which are in­de­pen­dent of B not in­crease in prob­a­bil­ity, they may ac­tu­ally de­crease in prob­a­bil­ity.

Here I may just be miss­ing the point but I don’t see how log­i­cal con­se­quences of A are rele­vant to the is­sue of whether in­duc­tion is oc­cur­ring.

I have to won­der if some strange no­tion of in­duc­tion is oc­cur­ring where in­tu­itions are sim­ply not shared. I won­der, what would hap­pen if we tabooed in­duc­tion?

• For any­one still fol­low­ing this, I have tried to restate my ar­gu­ments in a new way here:

http://​​www.crit­i­calra­tional­ism.net/​​2011/​​07/​​20/​​more-on-in­duc­tive-prob­a­bil­ity/​​

• There’s no ab­stract. It is not ob­vi­ous what is the point of the ar­ti­cle is.

• I linked to the pre­vi­ous post. That be­gins with some­thing like an ab­stract: a state­ment of in­tent, at least.

This isn’t an ar­ti­cle or a pa­per: it’s a blog post.

• After read­ing both of your posts I didn’t know what you meant by in­duc­tion nor what you were ar­gu­ing against.

In my re­sponse I didn’t ex­plic­itly urge you to stop us­ing the terms “in­duc­tion” and “sub­jec­tive in­ter­pre­ta­tion” (in an at­tempt to Re­place the Sym­bol with the Sub­stance through the game of Ta­boo Your Words). But I men­tioned that we might find we agree on ev­ery im­por­tant point af­ter strip­ping away the purely defi­ni­tional dis­putes. JoshuaZ and en­do­self made similar points here, though again they didn’t ex­plic­itly tell you how to clear up the con­fu­sion. I’m ask­ing you now to please find a new way of ex­press­ing your­self.

Un­til then I won’t know if this next part ac­tu­ally af­fects your ar­gu­ment, but it seems worth say­ing any­way: your math­e­mat­i­cal lemma does not deal with the sort of log­i­cal im­pli­ca­tions that sci­en­tists care about. If peo­ple fre­quently made state­ments such as, ‘The Stan­dard Model holds OR we won’t find the Higgs Bo­son on the 23rd of July 2011,’ then your lemma might seem like a perfectly nat­u­ral and in­tu­itive de­scrip­tion of ra­tio­nal de­grees of be­lief. In other words, your in­tu­ition may have mis­led you just be­cause in­tu­ition of­ten fails when it comes to math­e­mat­i­cal state­ments that we can’t in­ter­pret us­ing our life ex­pe­rience to date.

• Jaynes did point out some­thing about in­duc­tion which philoso­phers seem to miss: peo­ple perform in­duc­tion when the data they have gives us in­for­ma­tion about some sort of phys­i­cal mechanism that al­lows us to pre­dict the re­sults of that mechanism. For in­stance, if I have ob­served that 50 out of the last 50 times you have flipped a cer­tain coin, it has come up heads, I can con­clude that it’s prob­a­bly a two headed coin, and so I pre­dict that that coin will con­tinue to turn up only heads. If, how­ever, you pick up an­other quar­ter, in­duc­tion sud­denly be­comes a lot less pow­er­ful.

The other point Jaynes makes is that philoso­phers for­get the biggest ad­van­tage in­duc­tion gives us. Imag­ine this line of rea­son­ing: Fal­ling ap­ples obey New­to­nian me­chan­ics. The moon obeys New­to­nian me­chan­ics. The earth obeys New­to­nian me­chan­ics. Mars, Jupiter, and Saturn all be­have New­to­nian me­chan­ics. Nep­tune...does not. Hmm. Maybe there’s an­other planet? Oh, there’s Pluto. I guess Nep­tune was obey­ing New­to­nian me­chan­ics. Venus obeys New­to­nian me­chan­ics. Mer­cury...hmm. Mer­cury should not be mov­ing like that if the New­to­nian the­ory of grav­i­ta­tion is wrong. I guess we should in­ves­ti­gate this.

Most sci­en­tific ad­vances have come from times where in­duc­tion fails. When in­duc­tion works, it means we haven’t learned any­thing new.

• Well, since I was hor­ribly wrong when I thought I saw a flaw in the math, let me in­stead look at the con­clu­sions, and maybe I won’t be hor­ribly wrong :D

if p(A|B) > p(A), then |p(A v B|B) – p(A v B)| > |p(A v ~B|B) – p(A v ~B)|

That is, to the ex­tent that B in­creases the prob­a­bil­ity of A, it does so by in­creas­ing the prob­a­bil­ity of A v B more than it de­creases the prob­a­bil­ity of A v ~B. How­ever, since A v B is a log­i­cal con­se­quence of B to be­gin with, the in­crease in prob­a­bil­ity is a purely de­duc­tive in­fer­ence.

This is not what the equa­tion says above. Yes, p(AvB|B)=1. But there’s an­other term on that side too: -p(AvB), which has to be men­tioned. If it could be ig­nored, then B would in­crease the prob­a­bil­ity of A for any choice of A and B!

How’s this: to the ex­tent that B in­creases the prob­a­bil­ity of A, p(A v ~B) - p(A v ~B|B) is less than 1 - p(A v B).

Not de­duc­tive, I know, but ac­cu­rate.

Not only do log­i­cal con­se­quences of A which are in­de­pen­dent of B not in­crease in prob­a­bil­ity, they may ac­tu­ally de­crease in prob­a­bil­ity.

This is a bit cir­cu­lar. If some­thing is re­ally in­de­pen­dent of B, it will not change at all if we con­di­tion on B. Nearly all things that are log­i­cal con­se­quences, though, aren’t in­de­pen­dent. Maybe the au­thor had some ex­am­ple in mind while writ­ing?

• That is, to the ex­tent that B in­creases the prob­a­bil­ity of A, it does so by in­creas­ing the prob­a­bil­ity of A v B more than it de­creases the prob­a­bil­ity of A v ~B. How­ever, since A v B is a log­i­cal con­se­quence of B to be­gin with, the in­crease in prob­a­bil­ity is a purely de­duc­tive in­fer­ence.

But the de­crease in prob­a­bil­ity of A v ~B is not “purely de­duc­tive” be­cause ~(A v ~B) is not a log­i­cal con­se­quence of B. So the net change in the prob­a­bil­ity of A is not en­tirely de­duc­tive.

EDIT: This at­tacks the ar­gu­ment on its own terms, but in fact I think the ar­gu­ment given does not define in­duc­tion well enough to say any­thing about it.

• Ey ac­knowl­edges that P(A|B) != P(A), ey just dis­putes that this can be called in­duc­tion. This seems pointless to me; Bayesi­ans can just define in­duc­tion as cases where P(A|B) != P(A) and avoid this kind of word game.

• Whoops—v is ap­par­ently in­tended to mean mean “or” or “union.” It’s just that the au­thor in­cor­rectly trans­lates P(A v B) & P(A v ~B) into P(A v B) + P(A v ~B).

• Hi,

I am the au­thor. It wasn’t a mis­trans­la­tion. The log­i­cal equiv­alence was not trans­lated into any­thing. It was merely in­tended to break down A ac­cord­ing to its log­i­cal con­se­quences shared with B. I never wrote “P(A v B) + P(A v ~B),” be­cause that would be ir­rele­vant.

• In the very next equa­tion af­ter “A = (A v B) & (A v ~B)”, you write:

p(A v B|B) – p(A v B) + p(A v ~B|B) – p(A v ~B) = .15

This is the equa­tion where you put in the plus signs. Ad­di­tion­ally, you can break things down like that in­side the P() op­er­a­tor, but you can’t just move that to out­side the P() op­er­a­tor, be­cause things might be cor­re­lated (and, in the case of B and ~B, cer­tainly are).

• Well, it wasn’t ac­tu­ally an equa­tion. That’s why I used the =||= sym­bol. It was a bi­en­tail­ment. It as­serts log­i­cal equiv­alence (in clas­si­cal logic), and it means some­thing slightly differ­ent than an equals sym­bol. The equa­tion with the plus signs and the log­i­cal equiv­alence shouldn’t be con­fused.

• I’m back and there’s been no re­sponse, so I’ll be spe­cific. Start­ing from

p(A v B|B) – p(A v B) + p(A v ~B|B) – p(A v ~B) = .15

Us­ing p(X v Y) = p(X) + p(Y) - p(XY), we get
.15 = p(A|B) + p(B|B) - p(AB|B) - p(A) - p(B) + p(AB) + p(A|B) + p(~B|B) - p(A~B|B) - p(A) - p(~B) + p(A~B)
= p(A|B) + 1 - p(A) - p(A) - p(B) + p(AB) + p(A|B) - p(A) - p(~B) + p(A~B)
= 2 p(A|B) − 2 p(A) = twice the thing you started from, which is bad.

• It looks like you sim­plified p(AB|B) as p(A), but in fact p(AB|B)=p(ABB)/​p(B)=p(AB)/​p(B)=p(A|B). (I made a similar mis­take ear­lier.)

I get p(A|B) + p(B|B) - p(AB|B) - p(A) - p(B) + p(AB) + p(A|B) + p(~B|B) - p(A~B|B) - p(A) - p(~B) + p(A~B)

= p(A|B) + 1 - p(A|B) - p(A) - p(B) + p(AB) + p(A|B) + 0 − 0 - p(A) − 1 + p(B) + p(A~B)

= - p(A) - p(B) + p(AB) + p(A|B) - p(A) + p(B) + p(A~B)

= - p(A) + p(AB) + p(A|B) - p(A) + p(A~B)

= p(A|B) −2p(A) +p(AB) + p(A~B)

= p(A|B) −2p(A) + p(A)

= p(A|B) - p(A)

But this is quod erat demon­stran­dum.

• Okay. Yay, a use for the re­tract but­ton where it’s still good to have the text visi­ble!

• Man­fred,

I calcu­lated the re­sult for about three differ­ent sets of prob­a­bil­ities be­fore mak­ing the origi­nal post. The equa­tion was cor­rect each time. I could have just been mis­taken, of course, but even Zack (the com­menter above) con­ceded that the equa­tion is true.

EDIT: Oh, I see now. You have changed all my dis­junc­tions into con­junc­tions. Why?

• Ah, I’m sorry. Do you agree that the equa­tion I quoted above is in­cor­rect, though? I’m go­ing to have to leave now, but the rele­vant ba­sic equa­tions of prob­a­bil­ity are P(X v Y) = P(X) + P(Y) - P(X&Y), and P(X&Y) = P(X) * P(X | Y)

• I don’t know what this is try­ing to mean: “On the in­duc­tive view of prob­a­bil­is­tic in­fer­ence, B is am­plified to im­ply that A is more prob­a­bly true. This would would mean the log­i­cal con­se­quences of A which are not also log­i­cal con­se­quences of B should be more prob­a­ble given B.”

Edit: there was ran­dom other stuff here. Ig­nore it, I’m con­fused about the above.

• [Cor­rec­tion: In the origi­nal ver­sion of this com­ment, I claimed that the linked post was mis­taken on a point of math­e­mat­ics: speci­fi­cally, I said that it is not the case that P(A|B) – P(A)= P(A v B|B) – P(A v B) + P(A v ~B|B) – P(A v ~B). How­ever, Guy Srini­vasan pointed out that my sup­posed dis­proof con­tained a mis­take. Re­do­ing my calcu­la­tions, I find that the equa­tion in ques­tion is in fact an iden­tity. I re­gret the er­ror.]

• P(Av~B|B) does not equal P(A). P((Av~B) & B) equals P(A).

Edit: it doesn’t, of course. P(Av~B|B) = P(A|B) and the other thing I said is just silly.

• P(Av~B|B) does not equal P(A) [...] P(Av~B|B) = P(A|B)

Whoops! You’re right. I’ll edit the com­ment to re­flect this cor­rec­tion.

• Just for gen­eral con­ve­nience: P((A+~B)B) = P(AB).

• Since prob­a­bil­ity the­ory in gen­eral, and Bayes in par­tic­u­lar is of­ten seen as res­cu­ing in­duc­tion from the stan­dard ob­jec­tions, the ar­gu­ment is sig­nifi­cant.

I don’t see how that fol­lows. Is weight be­ing as­signed as a re­sult of Pop­per’s rep­u­ta­tion?

• A =||= (A v B) & (A v ~B)

Was that sup­posed to be (A & B) v (A & ~B)? What it has is always true. Also, what is =||=?

If I can un­der­stand this cor­rectly, they’re say­ing that in­duc­tion is false be­cause the only ac­cu­racy in it is due to the hid­den Bayes-struc­ture. This is true of ev­ery­thing.

• DanielLC,

Hi, I am the au­thor.

The =||= just means bi­en­tail­ment. It’s short for,

A |= (A v B) & (A v ~B) and (A v B) & (A v ~B) |= A

Where |= means en­tail­ment or log­i­cal con­se­quence. =||= is analo­gous to a bi­con­di­tional.

The point is that each side of a bi­en­tail­ment is log­i­cally equiv­a­lent, but the break­down al­lows us to see how B al­ters the prob­a­bil­ity of differ­ent log­i­cal con­se­quences of A.

• Thank you, I think I un­der­stand most of it now. I don’t see (at this hour) where the ab­solute val­ues come from, but that doesn’t seem to mat­ter much. Let’s fo­cus on this line:

There­fore, in the sub­jec­tive in­ter­pre­ta­tion, given B, one should have in­creased con­fi­dence in both A and ~(A v ~B), but that is a flat con­tra­dic­tion with a prob­a­bil­ity of 0.

The con­junc­tion of those two does con­tra­dict it­self, and if you ac­tu­ally write out the prob­a­bil­ity of the con­tra­dic­tion—us­ing the stan­dard product rule p(CD)=p(D)p(C|D) rather than mul­ti­ply­ing their sep­a­rate prob­a­bil­ities p(D)p(C) to­gether—you’ll see that it always equals zero.

But each sep­a­rate claim (A, and B~A), can in­crease in prob­a­bil­ity pro­vided that they each take prob­a­bil­ity from some­where else, namely from ~B~A. I see no prob­lem with re­gard­ing this as an in­crease for our sub­jec­tive con­fi­dence in A and a sep­a­rate in­crease for B~A. Again, each grows by re­plac­ing an op­tion (or doubt) which no longer ex­ists for us. Some of that doubt-in-A sim­ply changed into a differ­ent form of doubt-in-A, but some of it changed into con­fi­dence. The to­tal doubt there­fore goes down even though one part in­creases.