Sub-tweeting because I don’t want to rain on a poor PhD student who should have been advised better, but: that paper about LLMs having a map of the world is perhaps what happens when a famous physicist wants to do AI research without caring to engage with the existing literature.
I haven’t looked into the paper in question yet, but I have been concerned about researchers taking old ideas about AI risk and looking to prove things that might not be there yet as an AI risk communication point. Then, being overconfident that it is there.
This is quite bad for making scientific progress in AI Safety and I urge AI Safety researchers to be vigilant about making overconfident claims and having old ideas leak too much into their research conclusions.
If incorrect and disproven, you are also setting yourself up to lose total credibility in the wider community.
New tweet about the world model (map) paper:
I haven’t looked into the paper in question yet, but I have been concerned about researchers taking old ideas about AI risk and looking to prove things that might not be there yet as an AI risk communication point. Then, being overconfident that it is there.
This is quite bad for making scientific progress in AI Safety and I urge AI Safety researchers to be vigilant about making overconfident claims and having old ideas leak too much into their research conclusions.
If incorrect and disproven, you are also setting yourself up to lose total credibility in the wider community.