You don’t have very many predictions judged, so I’m not sure how reliable your worry is—only 8-11 predictions for each decile seems quite possible to just get lucky. Assuming that’s not the case, you could try mechanically bumping up every prediction by 10% and see what happens.
Also I notice there are many sets of predictions of the the form X will happen in 1 month/2 months/1 year/… with a separate prediction for each time period. How are these scored? Since these types of predictions are highly correlated scoring them individually can cause people to appear over or under confident.
Yes, that seems like a good point. I’m especially worried that my long-term predictions will turn out to be much too overconfident. And presumably I won’t see that until at least a while has gone by.
You don’t have very many predictions judged, so I’m not sure how reliable your worry is—only 8-11 predictions for each decile seems quite possible to just get lucky. Assuming that’s not the case, you could try mechanically bumping up every prediction by 10% and see what happens.
Also I notice there are many sets of predictions of the the form X will happen in 1 month/2 months/1 year/… with a separate prediction for each time period. How are these scored? Since these types of predictions are highly correlated scoring them individually can cause people to appear over or under confident.
Is there any reason to think the sets won’t balance out eventually?
The will, for very large values of eventually.
Yes, that seems like a good point. I’m especially worried that my long-term predictions will turn out to be much too overconfident. And presumably I won’t see that until at least a while has gone by.