I wouldn’t treat competitive forecasters as a homogeneous group, but I also think basically everyone was surprised by the rate of progress on the MATH dataset. The main difference is that the better forecasters adjusted quickly after the first surprise and were mostly calibrated after.
If you are interested, I did a detailed analysis of different groups of forecasters here: https://bounded-regret.ghost.io/scoring-ml-forecasts-for-2023/
I wouldn’t treat competitive forecasters as a homogeneous group, but I also think basically everyone was surprised by the rate of progress on the MATH dataset. The main difference is that the better forecasters adjusted quickly after the first surprise and were mostly calibrated after.