Note: despite the different username, I’m the author of the handbook and a former CFAR staff member.
I disagree with this take as specifically outlined, even though I do think there’s a kernel of truth to it.
Mainly, I disagree with it because it presupposes that obviously the important thing to talk about is nuclear weapons!
I suspect that Phil is unaware that the vast majority of both CFAR staff and prolific LWers have indeed 100% passed the real version of his test, which is writing and contributing to the subject of existential risk, especially that from artificial intelligence.
Phil may disagree with the claim that nuclear weapons are something like third on the list, rather than the top item, but that doesn’t mean he’s right. And CFAR staff certainly clear the bar of “spending a lot of time focusing on what seems to them to be the actually most salient threat.”
I agree that if somebody seems to be willfully ignoring a salient threat, they have gaps in their rationality that should give you pause.
(I clicked through to see your other comments after disagreeing with one. Generally, I like your comments!)
I think that EA writers and culture are less “lost” than you think, on this axis. I think that most EA/rationalist/ex-risk-focused people in this subculture would basically agree with you that the knowledge explosion/recursive acceleration of technological development is the core problem, and when they talk about “AI safety” and so forth, they’re somewhat shorthanding this.
Like, I think most of the people around here are, in fact, worried about some of the products rolling off the end of the assembly line, but would also pretty much immediately concur with you that the assembly line itself is the root problem, or at least equally important.
I can’t actually speak for everybody, of course, but I think you might be docking people more points than you should.