I mostly agree, although I’m still mulling it and think the issue is more complicated than it appears. One nitpick:
“I know that what I’m saying probably sounds pretty ‘out there’ and understand if you don’t believe me, but I’ve thought about this hard, and I think...”
Personally these kind of qualifiers rarely do anything to allay my doubt, and can easily increase them. I prefer to see incredulity. For instance, when a scientist has an amazing result, rather than seeing that they fully believe it but recognizing it’s difficult for me to believe, I’d rather see them doubtful of their own conclusion but standing by it nonetheless because of the strength of the evidence.
“I know it’s hard to believe, but it’s likely an AI will kill us all in the future.”
could become
“It’s hard for me to come to terms with, but there doesn’t seem to be any natural safeguards preventing an AI from doing serious damage.”
Personally these kind of qualifiers rarely do anything to allay my doubt, and can easily increase them. I prefer to see incredulity. For instance, when a scientist has an amazing result, rather than seeing that they fully believe it but recognizing it’s difficult for me to believe, I’d rather see them doubtful of their own conclusion but standing by it nonetheless because of the strength of the evidence.
Sure, I totally agree with this—I prefer your formulation to my own. My point was just that there ought to be some disclaimer—the one that I suggested is a weak example.
Edit: Well, okay, actually I prefer:
“It took me a long time to come to terms with, but there don’t seem to be any natural safeguards preventing an AI from doing serious damage.”
If one has actually become convinced of a position, it sounds disingenuous to say that it’s hard for one to come to terms with at present, but any apparently absurd position should at some point have been hard to come to terms with.
Adding such a qualifier is a good caution against appearing to be placing oneself above the listener. It carries the message “I know how you must be feeling about these things, I’ve been there too.”
I mostly agree, although I’m still mulling it and think the issue is more complicated than it appears. One nitpick:
Personally these kind of qualifiers rarely do anything to allay my doubt, and can easily increase them. I prefer to see incredulity. For instance, when a scientist has an amazing result, rather than seeing that they fully believe it but recognizing it’s difficult for me to believe, I’d rather see them doubtful of their own conclusion but standing by it nonetheless because of the strength of the evidence.
“I know it’s hard to believe, but it’s likely an AI will kill us all in the future.”
could become
“It’s hard for me to come to terms with, but there doesn’t seem to be any natural safeguards preventing an AI from doing serious damage.”
Sure, I totally agree with this—I prefer your formulation to my own. My point was just that there ought to be some disclaimer—the one that I suggested is a weak example.
Edit: Well, okay, actually I prefer:
“It took me a long time to come to terms with, but there don’t seem to be any natural safeguards preventing an AI from doing serious damage.”
If one has actually become convinced of a position, it sounds disingenuous to say that it’s hard for one to come to terms with at present, but any apparently absurd position should at some point have been hard to come to terms with.
Adding such a qualifier is a good caution against appearing to be placing oneself above the listener. It carries the message “I know how you must be feeling about these things, I’ve been there too.”