Just looking at the 50% questions, here is how I would score 1) if either direction is an event rather than the default and 2) if either direction is probably preferred by Scott:
US unemployment to be lower at end of year than beginning: 50%
Neither direction is an event, Yes is preferred.
SpaceX successfully launches a reused rocket: 50%
Yes is an event, Yes is preferred.
California’s drought not officially declared over: 50%
No is an event, No is preferred.
At least one SSC post > 100,000 hits: 50%
Yes is an event, Yes is preferred.
UNSONG will get > 1,000,000 hits: 50%
Yes is an event, Yes is preferred.
UNSONG will not miss any updates: 50%
No is an event, Yes is preferred.
I will be involved in at least one published/accepted-to-publish research paper by the end of 2016: 50%
Yes is an event, Yes is preferred.
[Over] 10,000 Twitter followers by end of this year: 50%
Yes is an event, Yes is preferred.
I will not get any new girlfriends: 50%
No is an event, perhaps No is preferred.
I will score 95th percentile or above in next year’s PRITE: 50%
Yes is an event, Yes is preferred.
I will not have any inpatient rotations: 50%
No is an event, perhaps Yes is preferred.
I get at least one article published on a major site like Huffington Post or Vox or New Statesman or something: 50%
Yes is an event, Yes is preferred.
I don’t attend any weddings this year: 50%
No is an event, perhaps No is preferred.
Scott would know better than I do, and he also could have marked a subset that he actually cared about.
Including the “perhaps”es, I count that 7⁄12 happened in the preferred direction, and 5⁄11 of the events happened. With this small sample there’s no sign of optimism bias, and he’s also well-calibrated on whether a non-default event will happen. Obviously you’d want to do this with the full set of questions and not just the 50% ones to get a more meaningful sample size.
US unemployment to be lower at end of year than beginning: 50%
Neither direction is an event
Well, to be pedantic if the US unemployment was exactly the same at the end of the year as at the beginning the prediction as worded by Scott would be false, so it could be argued that Yes is an event. (But the same would apply if he had written “higher” instead of “lower”.)
Just looking at the 50% questions, here is how I would score 1) if either direction is an event rather than the default and 2) if either direction is probably preferred by Scott:
Neither direction is an event, Yes is preferred.
Yes is an event, Yes is preferred.
No is an event, No is preferred.
Yes is an event, Yes is preferred.
Yes is an event, Yes is preferred.
No is an event, Yes is preferred.
Yes is an event, Yes is preferred.
Yes is an event, Yes is preferred.
No is an event, perhaps No is preferred.
Yes is an event, Yes is preferred.
No is an event, perhaps Yes is preferred.
Yes is an event, Yes is preferred.
No is an event, perhaps No is preferred.
Scott would know better than I do, and he also could have marked a subset that he actually cared about.
Including the “perhaps”es, I count that 7⁄12 happened in the preferred direction, and 5⁄11 of the events happened. With this small sample there’s no sign of optimism bias, and he’s also well-calibrated on whether a non-default event will happen. Obviously you’d want to do this with the full set of questions and not just the 50% ones to get a more meaningful sample size.
Well, to be pedantic if the US unemployment was exactly the same at the end of the year as at the beginning the prediction as worded by Scott would be false, so it could be argued that Yes is an event. (But the same would apply if he had written “higher” instead of “lower”.)