I am skeptical of the claim that the research is unique and hasn’t been published anywhere, and I’d also really like to know the details regarding what they prompted the model with.
The whole co-scientist thing looks really weird. Look at the graph there. Am I misreading it, or people rated it just barely better than raw o1 outputs? How is that consistent with it apparently pulling all of these amazing discoveries out of the air?
Edit: Found (well, Grok 3 found) an article with some more details regarding Penadés’ work. Apparently they did publish a related finding, and did feed it into the AI co-scientist system.
Generalizing, my current take on it is that they – and probably all the other teams that are now reporting amazing results – fed the system a ton of clues regarding the answer, on top of implicitly pre-selecting the problems to be those where they already knew there’s a satisfying solution to be found.
Yeah, my general assumption in these situations is that the article is likely overstating things for a headline and reality is not so clear cut. Skepticism is definitely warranted.
I am skeptical of the claim that the research is unique and hasn’t been published anywhere, and I’d also really like to know the details regarding what they prompted the model with.
The whole co-scientist thing looks really weird. Look at the graph there. Am I misreading it, or people rated it just barely better than raw o1 outputs? How is that consistent with it apparently pulling all of these amazing discoveries out of the air?
Edit: Found (well, Grok 3 found) an article with some more details regarding Penadés’ work. Apparently they did publish a related finding, and did feed it into the AI co-scientist system.
Generalizing, my current take on it is that they – and probably all the other teams that are now reporting amazing results – fed the system a ton of clues regarding the answer, on top of implicitly pre-selecting the problems to be those where they already knew there’s a satisfying solution to be found.
Yeah, my general assumption in these situations is that the article is likely overstating things for a headline and reality is not so clear cut. Skepticism is definitely warranted.