I have had one-on-ones or interviewed dozens of students who want a career in AI safety. [...] But when you ask them why they care about AI safety they don’t provide a particularly coherent answer. So I get more specific: “Why should we think AI is an existential risk?” Again, incoherent answer.
I’d say the situation is even worse than that. I’ve recently had one-on-ones with researchers in AI safety governance/policy, and most of them didn’t seem to have engaged with the object-level arguments. I suspect the same holds for most technical AI safety researchers, but my sample is small.
I suspect it’s mainly the lack of object-level thinking about the problem that leads many orgs and researchers to a prioritization that seems miscalibrated to me. The majority focuses on misuse risks, rather than existential risks from loss-of-control scenarios, even though the expected impact is far greater.
I’d say the situation is even worse than that. I’ve recently had one-on-ones with researchers in AI safety governance/policy, and most of them didn’t seem to have engaged with the object-level arguments. I suspect the same holds for most technical AI safety researchers, but my sample is small.
I suspect it’s mainly the lack of object-level thinking about the problem that leads many orgs and researchers to a prioritization that seems miscalibrated to me. The majority focuses on misuse risks, rather than existential risks from loss-of-control scenarios, even though the expected impact is far greater.