GPT-N that you can prompt with “I am stuck with this transformer architecture trying to solve problem X”. GPT-N would be AIHHAI if it answers along the lines of “In this arXiv article, they used trick Z to solve problems similar to X. Have you considered implementing it?”, and using an implementation of Z would solve X >50% of the time.
I haven’t finished reading the post, but I found it worthwhile for this quote alone. This is the first description I’ve read of how GPT-N could be transformative. (Upon reflection this was super obvious and I’m embarrassed I didn’t think of it myself – I got so distracted with arguments about whether GPT could do anything ‘original’ that I missed this strategy)
I recently chatted with someone who described an older-researcher that had encyclopedic knowledge of AI papers (ML and otherwise), who had seen a ton of papers exploring various tools and tricks for solving ML problems. The older researcher had the experience of watching many younger researchers reinvent the same tricks over and over, and being like “that’s cool, but, like, there’s a better version of this idea published 20 years ago, which not only does what your algorithm does but also solves SubProblem X better than you”.
So it was interesting that the world is a bit bottlenecked on only having so many ‘living libraries’ who are able to keep track of the deluge of research, but that this is something I’d expect GPT to be good enough at to meaningfully help.
I haven’t finished reading the post, but I found it worthwhile for this quote alone. This is the first description I’ve read of how GPT-N could be transformative. (Upon reflection this was super obvious and I’m embarrassed I didn’t think of it myself – I got so distracted with arguments about whether GPT could do anything ‘original’ that I missed this strategy)
I recently chatted with someone who described an older-researcher that had encyclopedic knowledge of AI papers (ML and otherwise), who had seen a ton of papers exploring various tools and tricks for solving ML problems. The older researcher had the experience of watching many younger researchers reinvent the same tricks over and over, and being like “that’s cool, but, like, there’s a better version of this idea published 20 years ago, which not only does what your algorithm does but also solves SubProblem X better than you”.
So it was interesting that the world is a bit bottlenecked on only having so many ‘living libraries’ who are able to keep track of the deluge of research, but that this is something I’d expect GPT to be good enough at to meaningfully help.