My baseline assumption for this is that everyone always does badly on these kinds of predictions. This frequently includes highly trained defense analysts with access to privileged information sources.
I suspect, at a gut level, that this is mostly because markets and crowds work on publicly available information and national security is pretty much the foremost subject for inside-view information being kept from the public.
I saw people discussing forecasting success of this on twitter and people were saying that the intelligence agencies actually called this right. Does anyone know an easy link to what those agencies were saying?
My baseline assumption for this is that everyone always does badly on these kinds of predictions. This frequently includes highly trained defense analysts with access to privileged information sources.
I don’t think this answers the question. 8% is an awfully strong prediction. Your meta-point is public info. People who honor your meta-point could make a killing in markets as badly miscalibrated as this one. That would make it profitable for people to correct the markets.
How can we expect well-calibrated markets on questions with so few events? It appears to me there is a ceiling on how good we can expect calibration to be here, and it is much lower than the average Metaculus question.
Of course, reflecting on this suggests to me we should be able to take a serious crack at the answer after this one, with more to follow afterward.
My baseline assumption for this is that everyone always does badly on these kinds of predictions. This frequently includes highly trained defense analysts with access to privileged information sources.
I suspect, at a gut level, that this is mostly because markets and crowds work on publicly available information and national security is pretty much the foremost subject for inside-view information being kept from the public.
I saw people discussing forecasting success of this on twitter and people were saying that the intelligence agencies actually called this right. Does anyone know an easy link to what those agencies were saying?
Context: https://twitter.com/ClayGraubard/status/1496699988801433602?s=20&t=mQ8sAzMRppI8Pr44O38M3w
https://twitter.com/ClayGraubard/status/1496866236973658112?s=20&t=mQ8sAzMRppI8Pr44O38M3w
The thing about intelligence agencies is that they are really good at insider trading.
A broken clock is right twice a day. If you always predict war in a crisis, you’ll eventually be right.
They do not often predict war this publicly.
I don’t think this answers the question. 8% is an awfully strong prediction. Your meta-point is public info. People who honor your meta-point could make a killing in markets as badly miscalibrated as this one. That would make it profitable for people to correct the markets.
So the question stands:
WTF?
How can we expect well-calibrated markets on questions with so few events? It appears to me there is a ceiling on how good we can expect calibration to be here, and it is much lower than the average Metaculus question.
Of course, reflecting on this suggests to me we should be able to take a serious crack at the answer after this one, with more to follow afterward.