The biggest problem is distinguishing them from fake experts, and some people seem to think AI experts solve that problem, but I think people are starting to realize the nature of AI sycophancy a bit now.
Human sorting used to be a lot better, to the point where this wasn’t nearly as big of a problem. IQ testing in the workplace was legal until a judge ruled otherwise in 1971. LLMs don’t have the same regulations applied to them, and can be evaluated as much as we like. Certainly GPT-4o can convince reddit users that it’s discovered the key to the universe, but the benchmarks put that claim to rest.
Besides that, humans allocate their upskilling time to plenty of different things—networking, public speaking, and so on. There’s a tradeoff between being the world’s best engineer and getting yourself in a position to use that skill. Assuming the current paradigm of throwing hard RLVR problems at LLMs holds, the problem of an unskilled human that spends his/her studying time getting to know its future bosses and hiring managers doesn’t seem like it naturally carries over.
Human sorting used to be a lot better, to the point where this wasn’t nearly as big of a problem. IQ testing in the workplace was legal until a judge ruled otherwise in 1971. LLMs don’t have the same regulations applied to them, and can be evaluated as much as we like. Certainly GPT-4o can convince reddit users that it’s discovered the key to the universe, but the benchmarks put that claim to rest.
Besides that, humans allocate their upskilling time to plenty of different things—networking, public speaking, and so on. There’s a tradeoff between being the world’s best engineer and getting yourself in a position to use that skill. Assuming the current paradigm of throwing hard RLVR problems at LLMs holds, the problem of an unskilled human that spends his/her studying time getting to know its future bosses and hiring managers doesn’t seem like it naturally carries over.