Thank you—and I strongly endorse this answer. And now that you point this out, I realize that it should have been clear. I have speculated in the past that a large part of the value of Superforecasting is that there are people actually motivated to investigate and do the expensive updating I have also said that I’m unsure how worthwhile it is to pay for the time of the types of people who can superforecast. This seems like a clear case where it is worthwhile, if only it worked.
Given that, I think there’s a strong case that we need large rewards for early correct updates away from consensus, especially for very rare events. (In a case like COVID, the value of faster information is in the tens or hundreds of billions of dollars. A tiny fraction of that would be more than enough.) But the typical time-weighted forecast scores don’t account for heterogeneous update costs or give sufficient reward to figuring it out a day sooner than the average—though metaculus’s score and the scoring Ozzie Gooen has looked at are trying to do this better. This seems very worth more consideration.
Thank you—and I strongly endorse this answer. And now that you point this out, I realize that it should have been clear. I have speculated in the past that a large part of the value of Superforecasting is that there are people actually motivated to investigate and do the expensive updating I have also said that I’m unsure how worthwhile it is to pay for the time of the types of people who can superforecast. This seems like a clear case where it is worthwhile, if only it worked.
Given that, I think there’s a strong case that we need large rewards for early correct updates away from consensus, especially for very rare events. (In a case like COVID, the value of faster information is in the tens or hundreds of billions of dollars. A tiny fraction of that would be more than enough.) But the typical time-weighted forecast scores don’t account for heterogeneous update costs or give sufficient reward to figuring it out a day sooner than the average—though metaculus’s score and the scoring Ozzie Gooen has looked at are trying to do this better. This seems very worth more consideration.