PredictionBook.com—Track your calibration

Our hosts at Tri­cy­cle Devel­op­ments have cre­ated Pre­dic­tionBook.com, which lets you make pre­dic­tions and then track your cal­ibra­tion—see whether things you as­signed a 70% prob­a­bil­ity hap­pen 7 times out of 10.

The ma­jor challenge with a tool like this is (a) com­ing up with good short-term pre­dic­tions to track (b) main­tain­ing your will to keep on track­ing your­self even if the re­sults are dis­cour­ag­ing, as they prob­a­bly will be.

I think the main mo­ti­va­tion to ac­tu­ally use it, would be ra­tio­nal­ists challeng­ing each other to put a pre­dic­tion on the record and track the re­sults—I’m go­ing to try to re­mem­ber to do this the next time Michael Vas­sar says “X%” and I as­sign a differ­ent prob­a­bil­ity. (Vas­sar would have won quite a few points for his su­pe­rior pre­dic­tions of Sin­gu­lar­ity Sum­mit 2009 at­ten­dance—I was pes­simistic, Vas­sar was ac­cu­rate.)