One area where I’ve changed my mind on emergent capabilities is that I now think most emergent capabilities really were us not realizing how large the Internet truly was, and not realizing how much data GPT-3 and GPT-4 had.
The even more deflationary hypothesis is that most of the emergent capabilities were basically data-contamination.
Here’s an example of how easy it is to data-contaminate LLMs, where it’s very easy to give models near-perfect replications of questions in the test set:
AIME I 2025: A Cautionary Tale About Math Benchmarks and Data Contamination
AIME 2025 part I was conducted yesterday, and the scores of some language models are available here:
I have to say I was impressed, as I predicted the smaller distilled models would crash and burn, but they actually scored at a reasonable 25-50%.
That was surprising to me! Since these are new problems, not seen during training, right? I expected smaller models to barely score above 0%. It’s really hard to believe that a 1.5B model can solve pre-math olympiad problems when it can’t multiply 3-digit numbers. I was wrong, I guess.
I then used openai’s Deep Research to see if similar problems to those in AIME 2025 exist on the internet. And guess what? An identical problem to Q1 of AIME 2025 exists on Quora:
I haven’t checked beyond that because the freaking p-value is too low already. Problems near identical to the test set can be found online.
So, what—if anything—does this imply for Math benchmarks? And what does it imply for all the sudden hill climbing due to RL?
I’m not certain, and there is a reasonable argument that even if something in the train-set contains near-identical but not exact copies of test data, it’s still generalization. I am sympathetic to that. But, I also wouldn’t rule out that GRPO is amazing at sharpening memories along with math skills.
At the very least, the above show that data decontamination is hard.
Never ever underestimate the amount of stuff you can find online. Practically everything exists online.
Unfortunately, the fact that companies won’t open their datasets makes it way too hard to actually study the issue of data contamination systematically.
One area where I’ve changed my mind on emergent capabilities is that I now think most emergent capabilities really were us not realizing how large the Internet truly was, and not realizing how much data GPT-3 and GPT-4 had.
The even more deflationary hypothesis is that most of the emergent capabilities were basically data-contamination.
Here’s an example of how easy it is to data-contaminate LLMs, where it’s very easy to give models near-perfect replications of questions in the test set:
Unfortunately, the fact that companies won’t open their datasets makes it way too hard to actually study the issue of data contamination systematically.