I’m a bit confused about forecasting tournaments and would appreciate any comments:
Suppose you take part in such a tournament.
You could predict as accurately as you can and get a good score. But let’s say there are some other equally good forecasters in the tournament and it becomes a random draw who wins. On expectation, all forecasters of the same quality have the same forecasts. If there are many good forecasters, your chances of winning become very low.
However, you could include some outlier predictions in your predictions. Then you lower your expected accuracy, but you also incrase your chances of winning the tournament if these outlier probabilities come true.
Therefore, I would expect a lot of noise in the relation between forecasting quality and being a tournament winner.
My knowledge level: I read the metaculus FAQ a couple days ago
At least on metaculus the prize pool is distributed among everyone with good enough accuracy, rather than winner-takes-all. So it shouldn’t be affected by the (real) phenomenon that you are describing.
Thanks, good to know. So I assume there is an incentive difference between monetary incentives that can be distributed in such a way, and the incentive of being able to say that you won a tournament (maybe also as a job qualification).
I’m a bit confused about forecasting tournaments and would appreciate any comments:
Suppose you take part in such a tournament.
You could predict as accurately as you can and get a good score. But let’s say there are some other equally good forecasters in the tournament and it becomes a random draw who wins. On expectation, all forecasters of the same quality have the same forecasts. If there are many good forecasters, your chances of winning become very low.
However, you could include some outlier predictions in your predictions. Then you lower your expected accuracy, but you also incrase your chances of winning the tournament if these outlier probabilities come true.
Therefore, I would expect a lot of noise in the relation between forecasting quality and being a tournament winner.
My knowledge level: I read the metaculus FAQ a couple days ago
At least on metaculus the prize pool is distributed among everyone with good enough accuracy, rather than winner-takes-all. So it shouldn’t be affected by the (real) phenomenon that you are describing.
Thanks, good to know. So I assume there is an incentive difference between monetary incentives that can be distributed in such a way, and the incentive of being able to say that you won a tournament (maybe also as a job qualification).