Thanks, this is helpful! After reading this post I bought ChatGPT Plus and tried a question on Deep Research:
Please find literature reviews / meta-analyses on the best intensity at which to train HIIT (i.e. maximum sustainable speed vs. leaving some in the tank)
I got much worse results than you did:
ChatGPT misunderstood my question. Its response answered the question “is HIIT better than MICT for improving fitness”.
Even allowing that we’re talking about HIIT vs. MICT: I was previously aware of 3 meta-analyses on that question. ChatGPT cited none of those 3 and instead cited 7 other studies, 1 of which was hallucinated, and 3 of which were individual studies not meta-analyses.
It made some claims but did not say which claims came from which sources, and there are some claims that look like they couldn’t have been in any of the sources (but I didn’t go through all of them).
In fact my results are so much worse that I suspected I did something wrong.
Last week I asked something similar on Perplexity (I don’t have the chat log saved) and it correctly understood what I wanted, and it reported that there were no studies that answered my question. I believe Perplexity is correct because I also could not find any relevant studies on Google Scholar.
I looked through ChatGPT again and I figured out that I did in fact do it wrong. I found Deep Research by going to the “Explore GPTs” button in the top right, which AFAICT searches through custom modules made by 3rd parties. The OpenAI-brand Deep Research is accessed by clicking the “Deep research” button below the chat input text box.
Thanks, this is helpful! After reading this post I bought ChatGPT Plus and tried a question on Deep Research:
I got much worse results than you did:
ChatGPT misunderstood my question. Its response answered the question “is HIIT better than MICT for improving fitness”.
Even allowing that we’re talking about HIIT vs. MICT: I was previously aware of 3 meta-analyses on that question. ChatGPT cited none of those 3 and instead cited 7 other studies, 1 of which was hallucinated, and 3 of which were individual studies not meta-analyses.
It made some claims but did not say which claims came from which sources, and there are some claims that look like they couldn’t have been in any of the sources (but I didn’t go through all of them).
In fact my results are so much worse that I suspected I did something wrong.
Link to my chat: https://chatgpt.com/share/67e30421-8ef4-8011-973f-2b39f0ae58a4
Last week I asked something similar on Perplexity (I don’t have the chat log saved) and it correctly understood what I wanted, and it reported that there were no studies that answered my question. I believe Perplexity is correct because I also could not find any relevant studies on Google Scholar.
I looked through ChatGPT again and I figured out that I did in fact do it wrong. I found Deep Research by going to the “Explore GPTs” button in the top right, which AFAICT searches through custom modules made by 3rd parties. The OpenAI-brand Deep Research is accessed by clicking the “Deep research” button below the chat input text box.