# Bets and updating

Sup­pose the US pres­i­den­tial elec­tion is to­mor­row. You cur­rently as­sign a prob­a­bil­ity of 50% to each out­come. (We are ig­nor­ing the small pos­si­bil­ity that nei­ther of the main party can­di­dates will win).

A man ap­proaches you, and offers you a bet of \$10, at 2-1 odds. In other words, if can­di­date one wins, he pays you \$20, if can­di­date two wins, you pay him \$10.

Should you ac­cept this bet? What if the bet was for \$10000 in­stead? As­sume that your util­ity is lin­ear in dol­lars (or as­sume that the bet is for utilons in­stead, what­ever). If not, why not? Try to think about this be­fore read­ing on.

The an­swer is that it de­pends on your pri­ors—in par­tic­u­lar, it de­pends on how you in­ter­pret the ev­i­dence of be­ing offered the bet. In gen­eral, if some­one offers you a large bet on some out­come, it’s prob­a­bly safe to as­sume they have ac­cess to a rea­son­able amount of in­for­ma­tion about the out­come. Depend­ing on how much in­for­ma­tion your own prob­a­bil­ity es­ti­mate is based on, you should up­date to­wards the odds they offered you.

If you up­date too lit­tle, and ac­cept too many bets, you will lose a lot of money to peo­ple with bet­ter in­for­ma­tion than you. On the other hand, you can also go too far in the other di­rec­tion. If your re­sponse to be­ing offered a five-cent bet is to im­me­di­ately up­date to ac­cept their prob­a­bil­ities (and re­fuse the bet), you will be very easy to fool (al­though hard to ex­ploit by bet­ting).

Now sup­pose there are two su­per­in­tel­li­gences, Omega and Omicron. They are both ex­cel­lent at mod­el­ling both you and the pres­i­den­tial elec­tion. Omega has a strong prefer­ence for money, and a weak prefer­ence for hav­ing you be­lieve false things about the pres­i­den­tial elec­tion. Omicron has this swapped—it wants you to be­lieve that ac­tual out­come of the elec­tion (which it has pre­dicted) is ex­tremely un­likely, and has a weak prefer­ence for money.

Omega ex­e­cutes the fol­low­ing plan: It looks through a large num­ber of pos­si­ble bets, look­ing for ones that will give it a lot of money (ac­cord­ing to its pre­dicted out­come of the elec­tion). For each of them, it pre­dicts whether you will, if offered, take the bet, or sim­ply up­date your be­lief to be more ac­cu­rate (if the bet is such that Omega wins money, it must be “in the cor­rect di­rec­tion”). It finds the best bet you will take (if any), and offers you this bet.

Omicron does a similar thing, but in­stead looks for bets that you won’t take—bets which will in­stead cause you to up­date strongly in the wrong di­rec­tion, and not take the bet (since it doesn’t want to give you money). Again, it finds the best bet you won’t take (if any) and ap­proaches you.

You are ap­proached by some­one and offered a bet, al­though you don’t know if it’s Omega or Omicron. What should your policy be?

If you ac­cept a bet, the bet is likely to have come from Omega, and thus be ex­tremely costly for you. So you shouldn’t take the bet.

On the other hand, if you up­date strongly in the di­rec­tion of the bet, it most likely came from Omicron, and this means it’s prob­a­bly an up­date in the wrong di­rec­tion. So you shouldn’t up­date.

This leaves you in the per­plex­ing situ­a­tion of be­liev­ing that the bet is prob­a­bly ex­tremely good, but not want­ing to take it.

• If you up­date too lit­tle, and ac­cept too many bets, you will lose a lot of money to peo­ple with bet­ter in­for­ma­tion than you. On the other hand, you can also go too far in the other di­rec­tion. If your re­sponse to be­ing offered a five-cent bet is to im­me­di­ately up­date to ac­cept their prob­a­bil­ities (and re­fuse the bet), you will be very easy to fool (al­though hard to ex­ploit by bet­ting).

This, in­ci­den­tally, is a real bookie thing. They worry con­stantly about in­formed bet­tors mak­ing money off them, but they also want the ‘flow’ in or­der to up­date their odds. So there’s a con­stant cat-and-mouse be­tween the bet­ting syn­di­cates who have in­for­ma­tional edges of var­i­ous sorts, and the book­mak­ers, who try to limit the bet­tors to small amounts while then turn­ing around and up­dat­ing their odds based on the re­vealed in­for­ma­tion (to the bet­tors’ peren­nial frus­tra­tion, as the book­mak­ers can close ac­counts unilat­er­ally or seize win­nings).

• Ad­ver­sar­ial ev­i­dence (that which is speci­fi­cally crafted to make your be­liefs worse) is tricky. It still fits, but you need to ex­pand the cir­cle of be­liefs you’re up­dat­ing. What are your pri­ors that the bet/​ev­i­dence comes from Omicron or Omega?

• Your pro­cess of de­cid­ing what to do may at some point in­clude simu­lat­ing Omega and Omicron. If so, this means that when Omega and Omicron are simu­lat­ing you, they are now try­ing to solve the Halt­ing Prob­lem. I am skep­ti­cal that Omega or Omicron can solve the Halt­ing Prob­lem.