Logic as Probability

Fol­lowup To: Put­ting in the Numbers

Be­fore talk­ing about log­i­cal un­cer­tainty, our fi­nal topic is the re­la­tion­ship be­tween prob­a­bil­is­tic logic and clas­si­cal logic. A robot run­ning on prob­a­bil­is­tic logic stores prob­a­bil­ities of events, e.g. that the grass is wet out­side, P(wet), and then if they col­lect new ev­i­dence they up­date that prob­a­bil­ity to P(wet|ev­i­dence). Clas­si­cal logic robots, on the other hand, de­duce the truth of state­ments from ax­ioms and ob­ser­va­tions. Maybe our robot starts out not be­ing able to de­duce whether the grass is wet, but then they ob­serve that it is rain­ing, and so they use an ax­iom about rain caus­ing wet­ness to de­duce that “the grass is wet” is true.

Clas­si­cal logic re­lies on com­plete cer­tainty in its ax­ioms and ob­ser­va­tions, and makes com­pletely cer­tain de­duc­tions. This is un­re­al­is­tic when ap­plied to rain, but we’re go­ing to ap­ply this to (first or­der, for starters) math later, which a bet­ter fit for clas­si­cal logic.

The gen­eral pat­tern of the de­duc­tion “It’s rain­ing, and when it rains the grass is wet, there­fore the grass is wet” was modus po­nens: if ‘U im­plies R’ is true, and U is true, then R must be true. There is also modus tol­lens: if ‘U im­plies R’ is true, and R is false, then U has to be false too. Third, there is the law of non-con­tra­dic­tion: “It’s si­mul­ta­neously rain­ing and not-rain­ing out­side” is always false.

We can imag­ine a robot that does clas­si­cal logic as if it were writ­ing in a note­book. Ax­ioms are en­tered in the note­book at the start. Then our robot starts writ­ing down state­ments that can be de­duced by modus po­nens or modus tol­lens. Even­tu­ally, the note­book is filled with state­ments de­ducible from the ax­ioms. Mo­dus tol­lens and modus po­nens can be thought of as con­sis­tency con­di­tions that ap­ply to the con­tents of the note­book.

Do­ing math is one im­por­tant ap­pli­ca­tion of our clas­si­cal-logic robot. The robot can read from its note­book “If vari­able A is a num­ber, A=A+0” and “SS0 is a num­ber,” and then write down “SS0=SS0+0.”

Note that this re­quires the robot to in­ter­pret vari­able A differ­ently than sym­bol SS0. This is one of many up­grades we can make to the ba­sic robot so that it can in­ter­pret math more eas­ily. We also want to pro­gram in spe­cial re­sponses to sym­bols like ‘and’, so that if A and B are in the note­book our robot will write ‘A and B’, and if ‘A and B’ is in the note­book it will add in A and B. In this light, modus po­nens is just the robot hav­ing a pro­grammed re­sponse to the ‘im­plies’ sym­bol.

Cer­tainty about our ax­ioms is what lets us use clas­si­cal logic, but you can rep­re­sent com­plete cer­tainty in prob­a­bil­is­tic logic too, by the prob­a­bil­ities 1 and 0. Th­ese two meth­ods of rea­son­ing shouldn’t con­tra­dict each other—if a clas­si­cal logic robot can de­duce that it’s rain­ing out, a prob­a­bil­is­tic logic robot with the same in­for­ma­tion should as­sign P(rain)=1.

If it’s rain­ing out, then my grass is wet. In the lan­guage of prob­a­bil­ities, this is P(wet|rain)=1. If I look out­side and see rain, P(rain)=1, and then the product rule says that P(wet and rain) = P(rain)·P(wet|rain), and that’s equal to 1, so my grass must be wet too. Hey, that’s modus po­nens!

The rules of prob­a­bil­ity can also be­have like modus tol­lens (if P(B)=0, and P(B|A)=1, P(A)=0) and the law of the ex­cluded mid­dle (P(A|not-A)=0). Thus, when we’re com­pletely cer­tain, prob­a­bil­is­tic logic and clas­si­cal logic give the same an­swers.

There’s a very short way to prove this, which is that one of Cox’s desider­ata for how prob­a­bil­ities must be­have was “when you’re com­pletely cer­tain, your plau­si­bil­ities should satisfy the rules of clas­si­cal logic.”

In Foun­da­tions of Prob­a­bil­ity, I al­luded to the idea that we should be able to ap­ply prob­a­bil­ities to math. Dutch book ar­gu­ments work be­cause our robot must act as if it had prob­a­bil­ities in or­der to avoid los­ing money. Sav­age’s the­o­rem ap­plies be­cause the re­sults of our robot’s ac­tions might de­pend on math­e­mat­i­cal re­sults. Cox’s the­o­rem ap­plies be­cause be­liefs about math be­have like other be­liefs.

This is com­pletely cor­rect. Math fol­lows the rules of prob­a­bil­ity, and thus can be de­scribed with prob­a­bil­ities, be­cause clas­si­cal logic is the same as prob­a­bil­is­tic logic when you’re cer­tain.

We can even use this cor­re­spon­dence to figure out what num­bers the prob­a­bil­ities take on:

1 for ev­ery state­ment that fol­lows from the ax­ioms, 0 for their nega­tions.

This raises an is­sue: what about bet­ting on the last digit of the 3^^^3′th prime? We dragged prob­a­bil­ity into this mess be­cause it was sup­posed to help our robot stop try­ing to prove the an­swer and just bet as if P(last digit is 1)=1/​4. But it turns out that there is one true prob­a­bil­ity dis­tri­bu­tion over math­e­mat­i­cal state­ments, given the ax­ioms. The right dis­tri­bu­tion is ob­tained by straight­for­ward ap­pli­ca­tion of the product rule—never mind that it takes 4^^^3 steps—and if you de­vi­ate from the right dis­tri­bu­tion that means you vi­o­late the product rule at some point.

This is why log­i­cal un­cer­tainty is differ­ent. Even though our robot doesn’t have enough re­sources to find the right an­swer, us­ing log­i­cal un­cer­tainty vi­o­lates Sav­age’s the­o­rem and Cox’s the­o­rem. If we want our robot to act as if it has some “log­i­cal prob­a­bil­ity,” it’s go­ing to need a stranger sort of foun­da­tion.

Part of the se­quence Log­i­cal Uncertainty

Pre­vi­ous Post: Put­ting in the Numbers

Next post: Ap­proach­ing Log­i­cal Uncertainty