I’m having the same experience. Half of my cursor queries it starts off like this.
Gemini 3 can tell you this string without searching the web, which other models usually cannot, indicating that Gemini 3 was very likely trained on benchmark data, possibly the whole swath of benchmarks that rely on the BIG-bench string.
Think this is false. Gemini does incredibly well on SimpleQA which is like the best general knowledge test. And google is just cracked at pretraining which is what gets the knowledge into the model. Seems plausible it remembering the string is just a byproduct of this.
(gemini 3.0, 2.5pro, sonnet 4.5, gpt 5.1)
edit: anyone care to share why they so heavily downvoted the post? Its fair if you disagree with the claim about gemini3 strong general knowledge explaining it knowing the canary string and therefore disagree-downvote, but why downvote the post overall?
I’m having the same experience. Half of my cursor queries it starts off like this.
Think this is false. Gemini does incredibly well on SimpleQA which is like the best general knowledge test. And google is just cracked at pretraining which is what gets the knowledge into the model. Seems plausible it remembering the string is just a byproduct of this.
(gemini 3.0, 2.5pro, sonnet 4.5, gpt 5.1)
edit: anyone care to share why they so heavily downvoted the post? Its fair if you disagree with the claim about gemini3 strong general knowledge explaining it knowing the canary string and therefore disagree-downvote, but why downvote the post overall?
Doing well on SimpleQA could also be evidence of benchmark contamination in the training data.
The “impossible scenario of actual time travel” [forward in time] is pretty funny. Thanks for replicating the key findings from the post.