Searching for Bayes-Structure

Gnomish helms should not func­tion. Their very con­struc­tion seems to defy the na­ture of thau­matur­gi­cal law. In fact, they are im­pos­si­ble. Like most prod­ucts of gnomish minds, they in­clude a large num­ber of bells and whis­tles, and very lit­tle sub­stance. Those that work usu­ally have a minor helm con­tained within, always hid­den away, dis­guised to ap­pear in­nocu­ous and inessen­tial.”
-- Spel­l­jam­mer cam­paign set

We have seen that knowl­edge im­plies mu­tual in­for­ma­tion be­tween a mind and its en­vi­ron­ment, and we have seen that this mu­tual in­for­ma­tion is ne­gen­tropy in a very phys­i­cal sense: If you know where molecules are and how fast they’re mov­ing, you can turn heat into work via a Maxwell’s De­mon /​ Szilard en­g­ine.

We have seen that form­ing true be­liefs with­out ev­i­dence is the same sort of im­prob­a­bil­ity as a hot glass of wa­ter spon­ta­neously re­or­ga­niz­ing into ice cubes and elec­tric­ity. Ra­tion­al­ity takes “work” in a ther­mo­dy­namic sense, not just the sense of men­tal effort; minds have to ra­di­ate heat if they are not perfectly effi­cient. This cog­ni­tive work is gov­erned by prob­a­bil­ity the­ory, of which ther­mo­dy­nam­ics is a spe­cial case. (Statis­ti­cal me­chan­ics is a spe­cial case of statis­tics.)

If you saw a ma­chine con­tinu­ally spin­ning a wheel, ap­par­ently with­out be­ing plugged into a wall out­let or any other source of power, then you would look for a hid­den bat­tery, or a nearby broad­cast power source—some­thing to ex­plain the work be­ing done, with­out vi­o­lat­ing the laws of physics.

So if a mind is ar­riv­ing at true be­liefs, and we as­sume that the sec­ond law of ther­mo­dy­nam­ics has not been vi­o­lated, that mind must be do­ing some­thing at least vaguely Bayesian—at least one pro­cess with a sort-of Bayesian struc­ture some­where—or it couldn’t pos­si­bly work.

In the be­gin­ning, at time T=0, a mind has no mu­tual in­for­ma­tion with a sub­sys­tem S in its en­vi­ron­ment. At time T=1,the mind has 10 bits of mu­tual in­for­ma­tion with S. Some­where in be­tween, the mind must have en­coun­tered ev­i­dence—un­der the Bayesian defi­ni­tion of ev­i­dence, be­cause all Bayesian ev­i­dence is mu­tual in­for­ma­tion and all mu­tual in­for­ma­tion is Bayesian ev­i­dence, they are just differ­ent ways of look­ing at it—and pro­cessed at least some of that ev­i­dence, how­ever in­effi­ciently, in the right di­rec­tion ac­cord­ing to Bayes on at least some oc­ca­sions. The mind must have moved in har­mony with the Bayes at least a lit­tle, some­where along the line—ei­ther that or vi­o­lated the sec­ond law of ther­mo­dy­nam­ics by cre­at­ing mu­tual in­for­ma­tion from noth­ing­ness.

In fact, any part of a cog­ni­tive pro­cess that con­tributes use­fully to truth-find­ing must have at least a lit­tle Bayesian struc­ture—must har­mo­nize with Bayes, at some point or an­other—must par­tially con­form with the Bayesian flow, how­ever nois­ily—de­spite how­ever many dis­guis­ing bells and whis­tles—even if this Bayesian struc­ture is only ap­par­ent in the con­text of sur­round­ing pro­cesses. Or it couldn’t even help.

How philoso­phers pon­dered the na­ture of words! All the ink spent on the true defi­ni­tions of words, and the true mean­ing of defi­ni­tions, and the true mean­ing of mean­ing! What col­lec­tions of gears and wheels they built, in their ex­pla­na­tions! And all along, it was a dis­guised form of Bayesian in­fer­ence!

I was ac­tu­ally a bit dis­ap­pointed that no one in the au­di­ence jumped up and said: “Yes! Yes, that’s it! Of course! It was re­ally Bayes all along!

But per­haps it is not quite as ex­cit­ing to see some­thing that doesn’t look Bayesian on the sur­face, re­vealed as Bayes wear­ing a clever dis­guise, if: (a) you don’t un­ravel the mys­tery your­self, but read about some­one else do­ing it (New­ton had more fun than most stu­dents tak­ing calcu­lus), and (b) you don’t re­al­ize that search­ing for the hid­den Bayes-struc­ture is this huge, difficult, om­nip­re­sent quest, like search­ing for the Holy Grail.

It’s a differ­ent quest for each facet of cog­ni­tion, but the Grail always turns out to be the same. It has to be the right Grail, though—and the en­tire Grail, with­out any parts miss­ing—and so each time you have to go on the quest look­ing for a full an­swer what­ever form it may take, rather than try­ing to ar­tifi­cially con­struct vaguely hand-wav­ing Grailish ar­gu­ments. Then you always find the same Holy Grail at the end.

It was pre­vi­ously pointed out to me that I might be los­ing some of my read­ers with the long es­says, be­cause I hadn’t “made it clear where I was go­ing”...

...but it’s not so easy to just tell peo­ple where you’re go­ing, when you’re go­ing some­where like that.

It’s not very helpful to merely know that a form of cog­ni­tion is Bayesian, if you don’t know how it is Bayesian. If you can’t see the de­tailed flow of prob­a­bil­ity, you have noth­ing but a pass­word - or, a bit more char­i­ta­bly, a hint at the form an an­swer would take; but cer­tainly not an an­swer. That’s why there’s a Grand Quest for the Hid­den Bayes-Struc­ture, rather than be­ing done when you say “Bayes!” Bayes-struc­ture can be buried un­der all kinds of dis­guies, hid­den be­hind thick­ets of wheels and gears, ob­scured by bells and whis­tles.

The way you be­gin to grasp the Quest for the Holy Bayes is that you learn about cog­ni­tive phe­nomenon XYZ, which seems re­ally use­ful—and there’s this bunch of philoso­phers who’ve been ar­gu­ing about its true na­ture for cen­turies, and they are still ar­gu­ing—and there’s a bunch of AI sci­en­tists try­ing to make a com­puter do it, but they can’t agree on the philos­o­phy ei­ther -

And—Huh, that’s odd! - this cog­ni­tive phe­nomenon didn’t look any­thing like Bayesian on the sur­face, but there’s this non-ob­vi­ous un­der­ly­ing struc­ture that has a Bayesian in­ter­pre­ta­tion - but wait, there’s still some use­ful work get­ting done that can’t be ex­plained in Bayesian terms—no wait, that’s Bayesian too—OH MY GOD this com­pletely differ­ent cog­ni­tive pro­cess, that also didn’t look Bayesian on the sur­face, ALSO HAS BAYESIAN STRUCTURE—hold on, are these non-Bayesian parts even do­ing any­thing?

  • Yes: Wow, those are Bayesian too!

  • No: Dear heav­ens, what a stupid de­sign. I could eat a bucket of amino acids and puke a bet­ter brain ar­chi­tec­ture than that.

Once this hap­pens to you a few times, you kinda pick up the rhythm. That’s what I’m talk­ing about here, the rhythm.

Try­ing to talk about the rhythm is like try­ing to dance about ar­chi­tec­ture.

This left me in a bit of a pickle when it came to try­ing to ex­plain in ad­vance where I was go­ing. I know from ex­pe­rience that if I say, “Bayes is the se­cret of the uni­verse,” some peo­ple may say “Yes! Bayes is the se­cret of the uni­verse!”; and oth­ers will snort and say, “How nar­row-minded you are; look at all these other ad-hoc but amaz­ingly use­ful meth­ods, like reg­u­larized lin­ear re­gres­sion, that I have in my toolbox.”

I hoped that with a spe­cific ex­am­ple in hand of “some­thing that doesn’t look all that Bayesian on the sur­face, but turns out to be Bayesian af­ter all”—and an ex­pla­na­tion of the differ­ence be­tween pass­words and knowl­edgeand an ex­pla­na­tion of the differ­ence be­tween tools and laws—maybe then I could con­vey such of the rhythm as can be un­der­stood with­out per­son­ally go­ing on the quest.

Of course this is not the full Se­cret of the Bayesian Con­spir­acy, but it’s all that I can con­vey at this point. Be­sides, the com­plete se­cret is known only to the Bayes Coun­cil, and if I told you, I’d have to hire you.

To see through the sur­face ad­hock­ery of a cog­ni­tive pro­cess, to the Bayesian struc­ture un­der­neath—to per­ceive the prob­a­bil­ity flows, and know how, not just know that, this cog­ni­tion too is Bayesian—as it always is—as it always must be - to be able to sense the Force un­der­ly­ing all cog­ni­tion—this, is the Bayes-Sight.

”...And the Queen of Kashfa sees with the Eye of the Ser­pent.”
“I don’t know that she sees with it,” I said. “She’s still re­cov­er­ing from the op­er­a­tion. But that’s an in­ter­est­ing thought. If she could see with it, what might she be­hold?”
“The clear, cold lines of eter­nity, I dare­say. Be­neath all Shadow.”
-- Roger Ze­lazny, Prince of Chaos