That might or might not be a better proxy for the kind of overconfidence I’ve been meaning to predict.
The reason why it might not: my formulation relied on the idea that most people will formulate their predictions such that the positive statement corresponds to the smaller subset of positive future space. In that case, even if it’s a < 50% prediction, I would still suspect it’s overconfident. For example:
6) South Korea and Philippines change alliance from USA to China and support it’s 9 dash line claims. Taiwan war with mainland China. 35%
Now I’ve no idea about the substance matter here, but across all such predictions, I predict that they’ll come true less often than the probability indicates. So if we use either of the methods you suggested here, the 35% figure moves upward rather than downward; however I think it should go down.
Fair enough! I suspect some low-probability predictions will be of that sort and some of the other, in which case there’s no simple way to adjust for overconfidence.
That might or might not be a better proxy for the kind of overconfidence I’ve been meaning to predict.
The reason why it might not: my formulation relied on the idea that most people will formulate their predictions such that the positive statement corresponds to the smaller subset of positive future space. In that case, even if it’s a < 50% prediction, I would still suspect it’s overconfident. For example:
Now I’ve no idea about the substance matter here, but across all such predictions, I predict that they’ll come true less often than the probability indicates. So if we use either of the methods you suggested here, the 35% figure moves upward rather than downward; however I think it should go down.
Fair enough! I suspect some low-probability predictions will be of that sort and some of the other, in which case there’s no simple way to adjust for overconfidence.