Vibe check: Metaculus’s track record on resolved AI questions seems worse than you would expect. I haven’t calculated any real scores, but there are many predictions that have gotten 50%+ for a while that resolve the other way. I mean naturally as predictions get closer to resolution without happening, their odds should go down, but guts tell me it still seems quite bad.
It’s not clear ultimately which direction it’s in. Forecasters seem to overestimate how much US politicians will care about AI and contest programming capabilities but simultaneously underestimate how much revenue will be generated by AI and MATH scores.
Vibe check: Metaculus’s track record on resolved AI questions seems worse than you would expect. I haven’t calculated any real scores, but there are many predictions that have gotten 50%+ for a while that resolve the other way. I mean naturally as predictions get closer to resolution without happening, their odds should go down, but guts tell me it still seems quite bad.
It’s not clear ultimately which direction it’s in. Forecasters seem to overestimate how much US politicians will care about AI and contest programming capabilities but simultaneously underestimate how much revenue will be generated by AI and MATH scores.