I encounter the same frustration when trying to talk about this. However, I think of it like this:
From the outside, arguing for AI risk looks highly analogous to evangelism.
You are concerned for the immortal soul (post singularity consciousness) of every human on earth, and that there is an imminent danger to them, posed by wildly powerful unfalsifiable unseen forces (yet-to-be-existent AGI), which cannot be addressed unless you get everyone you talk to to convert to your way of thinking (AI risk).
Or think about it like this: if an evangelist whose religion you disagree with takes every opportunity they get to try to convert you, how would you respond? If you wish to be kind to them, you sympathize with them while not taking their worldview seriously. And if they keep at it, your kindness might wear thin.
AI risk—or any potentially likely x-risk in general- is a very grabby memeplex. Taken seriously, if you think it’s at all a tractable problem, it warps your entire value system around it. Its resemblance to religion in this way is probably a major factor in giving many irreligious folk the ick when considering it. Most people who aren’t already bought into this are not down to have their entire value system upended like this, in a conversation where they likely already have epistemic guards up.
In my view, you often have to take a softer approach. Try to retreat to arguing for the sorts of background assumptions that naturally lead to x-risk concern, rather than arguing for it directly. Share the emotions that drive you, rather than the logic. Try not to push it- people can tell when you’re doing it. Appeal to the beliefs of people your interlocutor genuinely respects. And try to be vulnerable: if you aren’t willing to change your beliefs, why should they be? In conversation, minds are usually changed only when both parties have their guard down.
Doubtless there are darker arts of persuasion you could turn to. They could even be important or necessary. But I personally don’t want to think of myself as a soldier fighting for a cause, willing to use words as weapons. So this is what I’ve come to aspire to instead. It helps a lot in my own belief in the idea, also, to open it up to vulnerable critique from others; if I didn’t, I would just be letting the first grabby memeplex that saw me take me over, without fighting back. And, who knows—you might occasionally meet someone who shares enough assumptions to buy your worldview, and then at that point you can make the easier, logical case.
Edit: upon reflection, a lot of this applies to activism generally, too, not just religion. The connection to religion seems to me to be more prominent, however, due to its potential totality over all affairs of life, the way x-risk concern, and transhumanism more broadly, addresses the same kind of cosmic meaning religion often does, apocalyptic thinking, a central canon, set of institutions, and shared culture, organized gatherings explicitly framed as a secular surrogate for traditional holidays, and so on. Others have likely debated the particular distinctions elsewhere. The irreligionist in me wishes it weren’t so, but I feel I have to own the comparison if I want to see the world correctly. It’s been on my mind a lot.
you sympathize with them while not taking their worldview seriously
There is no reason at all to take any idea/worldview less than seriously. For the duration of engagement, be it 30 seconds as a topic comes up, or 30 minutes of a conversation, you can study anything in earnest. Better understanding, especially of the framing (which concerns are salient, how literal words translate into the issues they implicitly gesture at), doesn’t imply your beliefs or attitudes must shift as well.
if you aren’t willing to change your beliefs, why should they
This is not just an inadvisable or invalid principle, but with the epistemic sense of “belief” it’s essentially impossible to act this way. Beliefs explain and reflect reality, anything else is not a belief, so if you are changing your beliefs for any reason at all that is not about explaining and reflecting reality, they cease being beliefs in the epistemic sense, and become mental phenomena of some other nature.
There is no reason at all to take any idea/worldview less than seriously.
I think this is too absolute, at least for flawed humans as opposed to ideal rationalists. Some possible counterexamples:
The person talking to you is a skilled manipulator with potentially unfriendly goals
The idea is extremely distressing/depressing and you don’t feel psychologically safe engaging with it
The social cost of engaging with the idea outweighs the expected benefit of doing so
Certainly all three of those reasons can be misapplied; they are convenient excuses to protect one’s own flawed worldview, hang on to comforting delusions, or toe the line on politically charged issues. But sometimes doing those things really is better than the alternative.
Maybe there are modes of engagement that should be avoided, and many ideas/worldviews themselves are not worth engaging with (though neglectedness in your own personal understanding is a reason to seek them out). But as long as you have allocated time to something, even largely as a result of external circumstances, doing a superficial and half-hearted job of it is a waste. It certainly shouldn’t be the intent from the outset, as in the quote I was replying to.
Thanks for these reflections! Just one small clarification:
You are right that “concern for the immortal soul (post singularity consciousness) of every human on earth” may be off-putting to normies, and that such proclamations are best avoided in favor of more down-to-Earth considerations. And while my AI xrisk concern used to have quite a bit of that kind of utopian component, short AI timelines has shifted my point of view considerably, and I now worry mainly about the diminishing prospects for me, my loved ones and the remaining eight billion highly mortal humans alive today to make it into the 2030s and 2040s.
I encounter the same frustration when trying to talk about this. However, I think of it like this:
From the outside, arguing for AI risk looks highly analogous to evangelism.
You are concerned for the immortal soul (post singularity consciousness) of every human on earth, and that there is an imminent danger to them, posed by wildly powerful unfalsifiable unseen forces (yet-to-be-existent AGI), which cannot be addressed unless you get everyone you talk to to convert to your way of thinking (AI risk).
Or think about it like this: if an evangelist whose religion you disagree with takes every opportunity they get to try to convert you, how would you respond? If you wish to be kind to them, you sympathize with them while not taking their worldview seriously. And if they keep at it, your kindness might wear thin.
AI risk—or any potentially likely x-risk in general- is a very grabby memeplex. Taken seriously, if you think it’s at all a tractable problem, it warps your entire value system around it. Its resemblance to religion in this way is probably a major factor in giving many irreligious folk the ick when considering it. Most people who aren’t already bought into this are not down to have their entire value system upended like this, in a conversation where they likely already have epistemic guards up.
In my view, you often have to take a softer approach. Try to retreat to arguing for the sorts of background assumptions that naturally lead to x-risk concern, rather than arguing for it directly. Share the emotions that drive you, rather than the logic. Try not to push it- people can tell when you’re doing it. Appeal to the beliefs of people your interlocutor genuinely respects. And try to be vulnerable: if you aren’t willing to change your beliefs, why should they be? In conversation, minds are usually changed only when both parties have their guard down.
Doubtless there are darker arts of persuasion you could turn to. They could even be important or necessary. But I personally don’t want to think of myself as a soldier fighting for a cause, willing to use words as weapons. So this is what I’ve come to aspire to instead. It helps a lot in my own belief in the idea, also, to open it up to vulnerable critique from others; if I didn’t, I would just be letting the first grabby memeplex that saw me take me over, without fighting back. And, who knows—you might occasionally meet someone who shares enough assumptions to buy your worldview, and then at that point you can make the easier, logical case.
Edit: upon reflection, a lot of this applies to activism generally, too, not just religion. The connection to religion seems to me to be more prominent, however, due to its potential totality over all affairs of life, the way x-risk concern, and transhumanism more broadly, addresses the same kind of cosmic meaning religion often does, apocalyptic thinking, a central canon, set of institutions, and shared culture, organized gatherings explicitly framed as a secular surrogate for traditional holidays, and so on. Others have likely debated the particular distinctions elsewhere. The irreligionist in me wishes it weren’t so, but I feel I have to own the comparison if I want to see the world correctly. It’s been on my mind a lot.
There is no reason at all to take any idea/worldview less than seriously. For the duration of engagement, be it 30 seconds as a topic comes up, or 30 minutes of a conversation, you can study anything in earnest. Better understanding, especially of the framing (which concerns are salient, how literal words translate into the issues they implicitly gesture at), doesn’t imply your beliefs or attitudes must shift as well.
This is not just an inadvisable or invalid principle, but with the epistemic sense of “belief” it’s essentially impossible to act this way. Beliefs explain and reflect reality, anything else is not a belief, so if you are changing your beliefs for any reason at all that is not about explaining and reflecting reality, they cease being beliefs in the epistemic sense, and become mental phenomena of some other nature.
I think this is too absolute, at least for flawed humans as opposed to ideal rationalists. Some possible counterexamples:
The person talking to you is a skilled manipulator with potentially unfriendly goals
The idea is extremely distressing/depressing and you don’t feel psychologically safe engaging with it
The social cost of engaging with the idea outweighs the expected benefit of doing so
Certainly all three of those reasons can be misapplied; they are convenient excuses to protect one’s own flawed worldview, hang on to comforting delusions, or toe the line on politically charged issues. But sometimes doing those things really is better than the alternative.
Maybe there are modes of engagement that should be avoided, and many ideas/worldviews themselves are not worth engaging with (though neglectedness in your own personal understanding is a reason to seek them out). But as long as you have allocated time to something, even largely as a result of external circumstances, doing a superficial and half-hearted job of it is a waste. It certainly shouldn’t be the intent from the outset, as in the quote I was replying to.
Thanks for these reflections! Just one small clarification:
You are right that “concern for the immortal soul (post singularity consciousness) of every human on earth” may be off-putting to normies, and that such proclamations are best avoided in favor of more down-to-Earth considerations. And while my AI xrisk concern used to have quite a bit of that kind of utopian component, short AI timelines has shifted my point of view considerably, and I now worry mainly about the diminishing prospects for me, my loved ones and the remaining eight billion highly mortal humans alive today to make it into the 2030s and 2040s.