I think it is clear that people will use LLMs for autoresearch to the extent possible. The challenges there are reward hacking (as ever) and “research taste” (e.g., somehow predicting productive research directions beforehand, ignoring unproductive directions).
But if the data doesn’t exist, there is nothing the LLM can do. I don’t think there will be traditional looking labs full of human shaped robots (honestly, human shaped robots are largely a massive waste of time and resources), but air-conditioned warehouses full of “research cubes” able to control experimental conditions with great precision and being fed various reagents, but in the end these are external sensors and actuators collecting real world data by sampling it, so they are lab robots. I think the main problem here is that no matter how much better they are than humans, they are still very expensive per experiment in terms of time and resources, so there will be an LLM driven effort to construct synthetic data spaces that are seeded and verified using the real world experiments. This is a fairly major challenge on its own.
I think it is clear that people will use LLMs for autoresearch to the extent possible. The challenges there are reward hacking (as ever) and “research taste” (e.g., somehow predicting productive research directions beforehand, ignoring unproductive directions).
But if the data doesn’t exist, there is nothing the LLM can do. I don’t think there will be traditional looking labs full of human shaped robots (honestly, human shaped robots are largely a massive waste of time and resources), but air-conditioned warehouses full of “research cubes” able to control experimental conditions with great precision and being fed various reagents, but in the end these are external sensors and actuators collecting real world data by sampling it, so they are lab robots. I think the main problem here is that no matter how much better they are than humans, they are still very expensive per experiment in terms of time and resources, so there will be an LLM driven effort to construct synthetic data spaces that are seeded and verified using the real world experiments. This is a fairly major challenge on its own.