The Bayesian Tyrant

Long ago and far away, there was a king­dom called Es­ti­mor in a broad green valley sur­rounded by tall grey moun­tains. It was an av­er­age king­dom in most re­spects, un­til the King read the works of Robin Han­son and Eliezer Yud­kowsy, and de­cided to in­sti­tute a Roy­al­ist Futarchy.

(This is a parable about the differ­ences be­tween Bayesian up­dates and log­i­cal in­duc­tion. See also: Rad­i­cal Prob­a­bil­ism.)

The setup was very sim­ple. It fol­lowed the futarchic motto, “Vote Values, But Bet Beliefs”—the only spe­cial con­sid­er­a­tion be­ing that there was just one vot­ing con­stituent (that be­ing the King). A bet­ting mar­ket would in­form the King of ev­ery­thing He needed to know in or­der to best serve His in­ter­ests and the in­ter­ests of Es­ti­mor (which were, of course, one and the same).

The Seer’s Hall—a build­ing pre­vi­ously de­voted to re­li­gious prophecy—was re­pur­posed to serve the needs of the new bet­ting mar­ket. (The old prophets were, of course, wel­come to par­ti­ci­pate—so long as they were will­ing to put money on the line.)

All went well at first. The new bet­ting mar­ket al­lowed the King to set the rev­enue-max­i­miz­ing tax rate, via the Laf­fer curve. An early suc­cess of the mar­ket was the fore­cast­ing of a grain short­age based on crop growth early in the sea­son, which al­lowed am­ple time for grain to be pur­chased from neigh­bor­ing lands.

Be­ing an ex­pert Bayesian Him­self, the King would of­ten wan­der the Seer’s Hall, ques­tion­ing and dis­cussing with the traders at the mar­ket. Some­times the King would be shocked by what he learned there. For ex­am­ple, many of the traders were calcu­lat­ing the Kelly bet­ting crite­rion to de­ter­mine how much to in­vest in a sin­gle bet. How­ever, they then pro­ceeded to in­vest only a set frac­tion of the Kelly amount (such as 80%). When ques­tioned, traders replied that they were hedg­ing against mis­takes in their own calcu­la­tions, or re­duc­ing volatility, or the like.

One day, the King no­ticed a man who would always run in and out of the Hall, mak­ing bets hastily. This man did par­tic­u­larly well at the bet­ting ta­bles—he ended the day with a visi­bly heavy purse. How­ever, when ques­tioned by The King as to the source of his good luck, the man had no an­swers. This man will sub­se­quently be referred to as the Fool.

The King or­dered spies to fol­low the Fool on his daily busi­ness. That evening, spies re­turned to re­port that The Fool was run­ning back and forth be­tween the Seer’s Hall and Dragon’s Foot, a lo­cal tav­ern. The Fool would con­sult bet­ting odds at Dragon’s Foot, and re­turn to the Seer’s Hall to bet us­ing those odds.

Ev­i­dently, Dragon’s Foot had be­come an un­li­censed gam­bling den. But were they truly do­ing bet­ter than the Seer’s Hall, so that this man could profit sim­ply by us­ing their in­for­ma­tion?

The King had the Fool brought in for ques­tion­ing. As it turned out, the Fool was turn­ing a profit by ar­bi­trage be­tween the two mar­kets: when­ever there was a differ­ence in prices, the Fool would bet in fa­vor at the lo­ca­tion where prices were low, and bet against at the lo­ca­tion where prices were high. In this way, he was mak­ing guaran­teed money.

The King was dis­gusted at this way of mak­ing money with­out bring­ing valuable in­for­ma­tion to the mar­ket. He or­dered all other gam­bling in the King­dom shut down, re­quiring it to all take place at the Seer’s Hall.

Soon af­ter that, the Fool showed his face again. Once again, he did well in the mar­ket. The King had his spiel fol­low the Fool, but this time, he went nowhere of sig­nifi­cance.

Ques­tion­ing the Fool a sec­ond time, he learned that this time the Fool was mak­ing use of cal­ibra­tion charts. The Fool would make metic­u­lous records of the true his­tor­i­cal fre­quency of events given their prob­a­bil­is­tic judge­ment—for ex­am­ple, he had recorded that when the mar­ket judges an event to be 90% prob­a­ble, that event ac­tu­ally oc­curs about 85% of the time. The Fool had made these records about in­di­vi­d­ual traders as well as the mar­ket as a whole, and would place bets ac­cord­ingly.

The King was once again dis­gusted by the way the Fool made money off of the mar­ket with­out con­tribut­ing any ex­ter­nal in­for­ma­tion. But this time, He felt that He needed a more sub­tle solu­tion to the prob­lem. Think­ing back to his first days of read­ing about Bayes’ Law, the King re­al­ized the huge gap be­tween His vi­sion of perfected rea­son­ing and the re­al­ity of the crowded, noisy, ir­ra­tional mar­ket. The iron law of the mar­ket was buy low sell high. It did not fol­low ra­tio­nal logic. The Fool had proved it: the in­di­vi­d­ual traders were poorly cal­ibrated, and so was the mar­ket it­self.

What the King needed to do was re­form the mar­ket, mak­ing it a more ra­tio­nal place.

And so it was that the King in­sti­tuted the Bayesian Law: all bets on the mar­ket are re­quired to be Kelly bets made on valid prob­a­bil­ity es­ti­mates. Valid prob­a­bil­ity es­ti­mates are re­quired to be Bayesian up­dates of pre­vi­ously reg­istered prob­a­bil­ity es­ti­mates.

All traders on the mar­ket would now pro­ceed ac­cord­ing to Bayes’ Law. They would pre-reg­ister their prob­a­bil­ity dis­tri­bu­tions, pre-spec­i­fy­ing what kind of in­for­ma­tion would up­date them, and by how much it would up­date them.

The new or­di­nance proved bur­den­some. Only a few traders con­tinued to visit the Seer’s Hall. They spent their days in metic­u­lous calcu­la­tion, up­dat­ing de­tailed prior mod­els of grain and weather with all the data which poured in.

Sur­pris­ingly, the Fool was amongst the hang­ers-on, and con­tinued to make a tidy profit, even to the point of driv­ing out some of the re­main­ing traders—they sim­ply couldn’t com­pete with him.

The King ex­am­ined the reg­istered prob­a­bil­ity dis­tri­bu­tion the Fool was us­ing. It proved puz­zling. The Fool’s en­tire prob­a­bil­ity dis­tri­bu­tion was based on num­bers which were to be posted to a par­tic­u­lar tree out by Mulberry road. Up­dat­ing on these num­bers, the Fool was some­how a tidy profit. But where were the num­bers com­ing from?

The King’s spies found that the num­bers were be­ing posted by a se­cre­tive group, whose meet­ings they were un­able to in­fil­trate.

The King had all the at­ten­dees ar­rested, ac­cus­ing them of run­ning an ille­gal gam­bling ring. The Fool was brought in for ques­tion­ing once more.

“But it wasn’t a gam­bling ring!” the Fool protested. “They merely got to­gether and com­piled odds for gam­bling. They were quite ad­dicted when the Bayesian Law shut their sort out of the Seer’s Hall, af­ter all. And I took those odds and used them to bet in the Seer’s Hall, perfectly legally.”

“And re­dis­tributed the win­nings?” ac­cused the King.

“As is only fair,” agreed the Fool. “But that is not gam­bling. I sim­ply paid them as con­sul­tants.”

“You took money from hon­est Bayesi­ans, and drove them out of my Hall!”

“As is the ad­van­tage of Bayesi­anism, no?” The Fool cocked an eye­brow. “The money flows to he who can give the best odds.”

“Take him away!” the king bel­lowed, wav­ing a hand for the guards.

At that mo­ment, the guards re­moved their helmets, re­veal­ing them­selves to be com­rades-in-arms with the Fool. The out­casts of the Seer’s Hall had fore­seen that the King would move against them, and with the power of Futarchy, had pre­pared well—they staged a blood­less rev­olu­tion that day.

The King, his fam­ily, and his most loyal staff were forced into ex­ile. They went to stay with a dis­tant cousin of the King, who ruled the na­tion Lu­dos, in the next valley over.

The King of Lu­dos had, upon see­ing Es­ti­mor’s suc­cess with pre­dic­tion mar­kets, set up His own. Un­like the Seer’s Hall of Es­ti­mor, that of Lu­dos con­tinued to thrive.

The King in Ex­ile asked his cousin: “What did I do wrong? All I wanted was to serve Es­ti­mor. The pre­dic­tion mar­ket worked so well at first. And I only tried to im­prove it.”

The King of Lu­dos sat in thought for a time, and then spoke. “Cousin, We can­not tell You that You did any­thing wrong. Revolu­tions will hap­pen. But We will say this: the many pre­dic­tion mar­kets of Lu­dos strengthen each other. Run­ners go back and forth be­tween them, prof­it­ing from ar­bi­trage, and this only makes them stronger. Cal­ibra­tion traders cor­rect any bias of the mar­ket, en­sur­ing un­bi­ased re­sults. You tried to out­law the ir­ra­tional at your mar­ket—but re­mem­ber, the wise gam­bler prof­its off the foolhardy gam­bler. Without any novices throw­ing away their money, none would profit.”

“But most of all, cousin, We think You lost sight of the power of bet­ting. It is a truth more fun­da­men­tal than Bayes’ Law that money will flow from the un­clever to the clever. You lost Your trust in that sys­tem. Even if You had en­forced Kelly bet­ting, but left it to each in­di­vi­d­ual trader to set his prob­a­bil­ity how­ever he liked—rather than up­dat­ing via Bayes’ Law alone—you would have been fine. If Bayes’ Law were truly the cor­rect way, money would have flowed to those who ex­cel­led at it. If not, then money would have flowed el­se­where. But in­stead you over­whelmed them with the bu­reau­cracy of Bayes—re­quiring them to record ev­ery lit­tle bit of in­for­ma­tion they used to reach a con­clu­sion.”

The ar­bi­trage be­tween differ­ent bet­ting halls rep­re­sented out­side view /​ mod­est episte­mol­ogy, try­ing to reach agree­ment be­tween differ­ent rea­son­ers. It’s a ques­tion­able thing to in­clude, in terms of the point I’m mak­ing, since this is not ex­actly a thing that hap­pens in log­i­cal in­duc­tion. How­ever, it fits in the alle­gory so well that I felt I couldn’t not in­clude it. One ar­gu­ment for the com­mon prior as­sump­tion (an as­sump­tion which un­der­pins the Au­mann Agree­ment The­o­rem, and is closely re­lated to mod­est-episte­mol­ogy ar­gu­ments) is that a bookie can Dutch Book any group of agents who do not have a com­mon prior, via perform­ing ar­bi­trage on their var­i­ous be­liefs.

[Edit: ac­tu­ally, what we can con­clude from the anal­ogy is that bets on differ­ent mar­kets should con­verge to the same thing if they ever pay out, which is also true in log­i­cal in­duc­tion.]

The cal­ibra­tion-chart idea, clearly, rep­re­sented cal­ibra­tion prop­er­ties.

The idea of the Bayesian Law rep­re­sented re­quiring all hy­pothe­ses/​traders to up­date in a Bayesian man­ner. Start­ing from Bayesian hy­poth­e­sis test­ing, one step we can take in the di­rec­tion of log­i­cal in­duc­tion is to al­low hy­pothe­ses to them­selves make non-Bayesian up­dates. The over­all up­date be­tween hy­pothe­ses would re­main Bayesian, but an in­di­vi­d­ual hy­poth­e­sis could change its mind in a non-Bayesian fash­ion. A hy­poth­e­sis would still be re­quired to have a co­her­ent prob­a­bil­ity dis­tri­bu­tion at any given time; just, the up­dates could be non-Bayesian. A fan of Bayes’ Law might sup­pose that, in such a situ­a­tion, the hy­pothe­ses which up­date ac­cord­ing to Bayes’ Law would dom­i­nate—in other words, a meta-level Bayesian would learn to also be an ob­ject-level Bayesian. But I see no rea­son to sus­pect this to be the case. In­deed, in situ­a­tions where log­i­cal un­cer­tainty is rele­vant, non-Bayesian up­dates can be used to con­tinue im­prov­ing one’s prob­a­bil­ity dis­tri­bu­tion over and above the ex­plicit ev­i­dence which comes in. It was this idea—that we could take one step to­ward log­i­cal in­duc­tion by be­ing a meta-level Bayesian with­out be­ing an ob­ject-level Bayesian—which in­spired this post (al­though the alle­gory didn’t end up hav­ing such a strong con­nec­tion with this idea).

The main point of this post, any­way, is that Bayes’ Law would be a bad law. Don’t in­sti­tute a re­quire­ment that ev­ery­one rea­son ac­cord­ing to it.