In AI safety, this can be a serious problem.
I have had one-on-ones or interviewed dozens of students who want a career in AI safety. [...] But when you ask them why they care about AI safety they don’t provide a particularly coherent answer. So I get more specific: “Why should we think AI is an existential risk?” Again, incoherent answer.
I’d say the situation is even worse than that. I’ve recently had one-on-ones with researchers in AI safety governance/policy, and most of them didn’t seem to have engaged with the object-level arguments. I suspect the same holds for most technical AI safety researchers, but my sample is small.
I suspect it’s mainly the lack of object-level thinking about the problem that leads many orgs and researchers to a prioritization that seems miscalibrated to me. The majority focuses on misuse risks, rather than existential risks from loss-of-control scenarios, even though the expected impact is far greater.
I’ve never looked closely at the question of a CERN for AI. I feel like I’ve heard about it from afar, and then the topic mostly disappeared from my bubble, to be replaced by the idea of an IAEA for AI. I assumed that it was simply the initial intuition which, through organically refining itself within the community, had converged from CERN to IAEA for the reasons you mention in your post.
I imagine that what motivated you to write this post is that you somewhat regularly hear people defending the idea of a CERN for AI, so do you know whether the 3 visions a), b) and c) of the CERN for AI that you describe are defended by many people, and in what proportion?