I am sure he is not in denial. He knows that the AI systems are on the trajectory to the top and beyond.
Ok, denial is too strong a word. I don’t exactly know how to describe the mental motion he’s doing though.
By volume, his post thread is mostly discussions of ways in which this isn’t a fair comparison, whereas the correct epistemic update is more like “OK so competition maths is solved, what does this mean next?”. It’s a level of garymarcusing where he doesn’t disagree with any facts on the ground but the overall vibe of the piece totally misses the wood for the trees in a particular and consistent direction. Terry’s opinions on maths AI (which one would hope to be a useful data point) are being relegated to a lagging indicator by this mental motion.
I am sure we’ll see an easy and consistent 42 score from the models sooner rather than later, and we’ll see much more than that in the adjacent areas, but not yet :-)
(Someone who got a bronze in late 1960-s is telling me that this idea to give gold medals to 10+% of the participants is relatively recent, that when they were competing back in the 60-s there would be exactly 5 gold medals with this table of results.)
My recollection from the late 1980s when I was doing IMOs is that the proportions were supposed to be something like 6:3:2:1 nothing:bronze:silver:gold, so about 8% gold medals. I don’t think I ever actually verified this by talking to senior officials or checking the numbers.
(As for Terry Tao, I agree with you that he is clearly not in denial, he’s just cross at OpenAI for preferring PR over (1) good science and (2) politeness.)
Yeah, I actually looked at the early years today, and in 1969 only the three perfect scores won gold, and in 1970 this was relaxed a little bit, and the overall trend looked to me like there were multiple reforms with gradual relaxation of the standards for gold (although I did not do more than superficial sampling from several time points).
I think the official goal is still approximately 6:3:2:1, but this year those fuzzy boundaries resulted in 67 gold medals out of 630 participants (slightly above 10.6%).
Ok, denial is too strong a word. I don’t exactly know how to describe the mental motion he’s doing though.
By volume, his post thread is mostly discussions of ways in which this isn’t a fair comparison, whereas the correct epistemic update is more like “OK so competition maths is solved, what does this mean next?”. It’s a level of garymarcusing where he doesn’t disagree with any facts on the ground but the overall vibe of the piece totally misses the wood for the trees in a particular and consistent direction. Terry’s opinions on maths AI (which one would hope to be a useful data point) are being relegated to a lagging indicator by this mental motion.
I would not say it is solved :-)
I am sure we’ll see an easy and consistent 42 score from the models sooner rather than later, and we’ll see much more than that in the adjacent areas, but not yet :-)
(Someone who got a bronze in late 1960-s is telling me that this idea to give gold medals to 10+% of the participants is relatively recent, that when they were competing back in the 60-s there would be exactly 5 gold medals with this table of results.)
My recollection from the late 1980s when I was doing IMOs is that the proportions were supposed to be something like 6:3:2:1 nothing:bronze:silver:gold, so about 8% gold medals. I don’t think I ever actually verified this by talking to senior officials or checking the numbers.
(As for Terry Tao, I agree with you that he is clearly not in denial, he’s just cross at OpenAI for preferring PR over (1) good science and (2) politeness.)
Yeah, I actually looked at the early years today, and in 1969 only the three perfect scores won gold, and in 1970 this was relaxed a little bit, and the overall trend looked to me like there were multiple reforms with gradual relaxation of the standards for gold (although I did not do more than superficial sampling from several time points).
I think the official goal is still approximately 6:3:2:1, but this year those fuzzy boundaries resulted in 67 gold medals out of 630 participants (slightly above 10.6%).