A person new to AI safety evaluating their arguments is roughly at a similar position to a Go novice trying to make sense of two Go grandmasters disagreeing about a board
I don’t think the analogy is great, because Go grandmasters have actually played, lost and (critically) won a great many games of Go. This has two implications: first, I can easily check their claims of expertise. Second, they have had many chances to improve their gut level understanding of how to play the game of Go well, and this kind of thing seems to be to necessary to develop expertise.
How does one go about checking gut level intuitions about AI safety? It seems to me that turning gut intuitions into legible arguments that you and others can (relatively) easily check is one of the few tools we have, with objectively assessable predictions being another. Sure, both are hard, and it would be nice if we had easier ways to do it, but it seems to me that that’s just how it is.
In case of AI safety, the analogy maps through things like past research results, or general abilities to reason and make arguments. You can check the claim that e.g. Eliezer historically made many good non-trivial arguments about AI, where he was the first person, or one of the first people to make them. While the checking part is less easy than in chess, I would say it’s roughly comparable to high level math, or good philosophy.
I don’t think the analogy is great, because Go grandmasters have actually played, lost and (critically) won a great many games of Go. This has two implications: first, I can easily check their claims of expertise. Second, they have had many chances to improve their gut level understanding of how to play the game of Go well, and this kind of thing seems to be to necessary to develop expertise.
How does one go about checking gut level intuitions about AI safety? It seems to me that turning gut intuitions into legible arguments that you and others can (relatively) easily check is one of the few tools we have, with objectively assessable predictions being another. Sure, both are hard, and it would be nice if we had easier ways to do it, but it seems to me that that’s just how it is.
In case of AI safety, the analogy maps through things like past research results, or general abilities to reason and make arguments. You can check the claim that e.g. Eliezer historically made many good non-trivial arguments about AI, where he was the first person, or one of the first people to make them. While the checking part is less easy than in chess, I would say it’s roughly comparable to high level math, or good philosophy.