These are especially common, surprisingly perhaps, in AI and ML departments.
This is somewhat unsurprising given human psychology. - Scaling up LLMs killed a lot of research agendas inside ML, particularly NLP. Imagine your whole research career was built on improving benchmarks on some NLP problem using various clever ideas. Now, the whole thing is better solved by three sentence prompt to GPT4 and everything everyone in the subfield worked on is irrelevant for all practical purposes… how do you feel? In love with scaled LLMs? - Overall, people often like about research is coming up with smart ideas, and there is some aesthetics going into it. What’s traditionally not part of the aesthetics is ‘and you also need to get $100M in compute’, and it’s reasonably to model a lot of people as having a part which hates this.
Kinda like mathematicians hated it when the four color theorem was solved by a computer brute-forcing thousands of options. Only imagine that the same thing happens to hundreds of important mathematical problems—the proper way to solve them becomes to reduced them to a huge by finite number of cases, then throw lots of money at a computer who will handle these cases one by one, producing a “proof” that no human will ever be able to verify directly.
This is somewhat unsurprising given human psychology.
- Scaling up LLMs killed a lot of research agendas inside ML, particularly NLP. Imagine your whole research career was built on improving benchmarks on some NLP problem using various clever ideas. Now, the whole thing is better solved by three sentence prompt to GPT4 and everything everyone in the subfield worked on is irrelevant for all practical purposes… how do you feel? In love with scaled LLMs?
- Overall, people often like about research is coming up with smart ideas, and there is some aesthetics going into it. What’s traditionally not part of the aesthetics is ‘and you also need to get $100M in compute’, and it’s reasonably to model a lot of people as having a part which hates this.
Kinda like mathematicians hated it when the four color theorem was solved by a computer brute-forcing thousands of options. Only imagine that the same thing happens to hundreds of important mathematical problems—the proper way to solve them becomes to reduced them to a huge by finite number of cases, then throw lots of money at a computer who will handle these cases one by one, producing a “proof” that no human will ever be able to verify directly.