I wonder, have many people on here not read the many examples of how harmful certain types of personality can be, for instance in this post and this one?
I have strong-downvoted both of those posts! I think they do a mixture of the “taking propaganda at face value” thing, seem to largely be in some kind of negative affect spiral around imagining arbitrarily bad qualities they can assign to people, seem to take an enormously naive view of the psychopathology literature, seem to consistently avoid dealing with the moral reflection dynamics I outline in this post, and seem very rooted in negative utilitarianism, which I disagree with[1].
Overall, I am pretty sad about that whole set of stuff. I somewhat wish I could talk more to people about it, but the negative utilitarianism-leaning stuff usually makes good-faith discourse a lot harder (since the bottom-line for my interlocutors seems clearly written).
Like man, those posts are so frustrating to read. They just… assert things. No epistemic statuses, no caveats, just stuff like this:
The most ardent fanatics are utterly convinced they have found the one infallible authority in possession of ultimate truth and righteousness; they are textbook dogmatists (Rokeach, 1960). For religious fundamentalists, this is usually a holy book containing the divine revelation of God and his prophets. For Nazis, it was Hitler’s Führerprinzip (Leader Principle), codified by Rudolf Hess’s declaration that “the Führer is always right”. Similarly, many communist revolutionaries essentially placed absolute faith in foundational texts like Marx’s Das Kapital, or in the Party itself (Montefiore, 2007). “Angkar is an organization that cannot make mistakes” was a key slogan of the Khmer Rouge.[4]
For the fanatic, any doubt or deviation from these dogmas is not only wrong but evil, culminating in a total “soldier mindset” which defends the pre-existing ideology at all costs (Galef, 2021). This necessitates abandoning even the most basic form of empiricism by “rejecting the evidence of one’s own eyes and ears”, to paraphrase Orwell.[5] The fanatic is thus essentially incorrigible and has no epistemic or moral uncertainty, even in the face of widespread opposition (Gollwitzer et al., 2022).[6]
It honestly reads to me kind of like true crime podcasts? They love always describing everyone they cover with these kinds of extreme characterizations which basically never hold up when you look into them. My honest best guess is these posts serve a kind of similar role within our community.
I am confident the psychological characterization in paragraphs like the above is wrong. “Defends pre-existing ideology at all costs”. Come on, no, of course not “all costs”. Indeed a huge fraction of people caught up in these ideologies end up deconverting or drastically mellowing their views when their social context changes. These seem hilariously strawmanny absolutes that get ascribed to people here.
The fanatic is thus essentially incorrigible and has no epistemic or moral uncertainty, even in the face of widespread opposition
This is exactly the kind of thing I mean when I say people seem really got by propaganda, and when people engage in the fundamental attribution error. Like, I don’t know how I would differentiate paragraphs like the above from ravings of religious people talking about demons or devils.
It seems like in these posts all negative attributes must be assigned to these people, and all their cognitive leanings must be absolute and unwielding. They must “defend the pre-existing ideology at all costs”, they are “utterly convinced”, they are “textbook dogmatists”, they place “absolute faith” in the holy texts. Come on, this is not how people work.
Thanks for sharing that take, which I find largely quite bizarre and surprising. I continue to think those posts are super valuable.
I can understand finding the negative(-leaning) utilitarianism codedness of the writings annoying, but I don’t see why you think it makes good-faith discussion a lot harder. From my perspective, a lot of writings on LW are “yay, hurrah team human”-coded in a way that annoys the crap out of me and makes me want to punch things, but it’s not like that means I can’t get valuable things out of the writings or have to treat the posters here as necessarily adversarial.
I am confident the psychological characterization in paragraphs like the above is wrong. “Defends pre-existing ideology at all costs”. Come on, no, of course not “all costs”. Indeed a huge fraction of people caught up in these ideologies end up deconverting or drastically mellowing their views when their social context changes.
The subject of the sentence you put in quotation marks was “The fanatic”—as in, “the archetypal example of the fanatic.” This sounds to me like more of a writing style issue. Like, it wouldn’t even occur to me to assume that the post is saying that every believer of a harmful ideology is like that. More that there’s a fanaticism attractor and that people at the center of it really do approach that described extreme—which I think is true? I guess that’s the thing you’re contesting, but I feel like history contains some examples of atrocities that are hard to explain without choosing at least one of the following: either some people can get incredibly fanatical, or some people are sadistic/evil and may use fanaticism/ideology as a cover. Either way, at least one of the posts’ messages must be true?
There are some far-future relevance speculations where my model of what is likely to happen with AI is different from David’s, so I think it’s just much more likely that AIs will take over and it won’t matter what the humans who built it were like. But that doesn’t really invalidate too much—I mean part of the posts are also about what sort of qualities we wouldn’t want to see in AIs that we build. On your point about how AI-aided moral reflection and overcoming resource scarcity would reduce fanaticism or other bad consequences from “bad values,” I think that sort of point is underappreciated in some EA circles, but it’s not like it’s obvious that this is what’s going to happen. I think the posts we’re discussing engage well with reasons why reflection might not solve all the problems.
The subject of the sentence you put in quotation marks was “The fanatic”—as in, “the archetypal example of the fanatic.” This sounds to me like more of a writing style issue. Like, it wouldn’t even occur to me to assume that the post is saying that every believer of a harmful ideology is like that. More that there’s a fanaticism attractor and that people at the center of it really do approach that described extreme—which I think is true? I guess that’s the thing you’re contesting, but I feel like history contains some examples of atrocities that are hard to explain without choosing at least one of the following: either some people can get incredibly fanatical, or some people are sadistic/evil and may use fanaticism/ideology as a cover. Either way, at least one of the posts’ messages must be true?
The rest of the post then tries to argue that these kinds of mental traits are relatively widespread, and argues from that incorrigibility and absolute fanaticism.
I agree an “archetype” definition could be fine. But it’s IMO clearly not what’s going on. The post makes no attempt at clarifying how far of an outlier all of the above descriptions are, and later includes sections like:
In the US, around 20% of American adults (roughly 50 million) agree that “God has called Christians to exercise dominion over all areas of American society” (2023 PRRI/Brookings survey, p.4). Similarly, nearly a quarter of US adults (Pew Research Center, 2022) say the Bible should have “a great deal of influence” on US laws. Extrapolating data from a2008-2009 Pew survey (p.47) of 19 African countries, we estimate that roughly 15% of Africa’s 700 million Christians (roughly 100 million) believe that violence against civilians in defense of Christianity can often or sometimes be justified. Christians in Europe and Latin America may plausibly be less fanatical on average. Still, perhaps 200-250 million Christians worldwide (8-10%) could reasonably be classified as ideological fanatics.
This is obviously absurd! 200-250 million Christians absolutely do not fit the description that I quoted in my comment above, which is the only definition they give for what they mean by “ideological fanatics”:
For the fanatic, any doubt or deviation from these dogmas is not only wrong but evil, culminating in a total “soldier mindset” which defends the pre-existing ideology at all costs (Galef, 2021). This necessitates abandoning even the most basic form of empiricism by “rejecting the evidence of one’s own eyes and ears”, to paraphrase Orwell.[5] The fanatic is thus essentially incorrigible and has no epistemic or moral uncertainty, even in the face of widespread opposition (Gollwitzer et al., 2022).[6]
This is obviously not how this works. This is such a blatant example of category gerrymandering and the noncentral fallacy that I think my upset is very justified.
But ideological fanatics take such traits to extremes.
[...]
For the fanatic, any doubt or deviation from these dogmas is not only wrong but evil, culminating in a total “soldier mindset” which defends the pre-existing ideology at all costs (Galef, 2021). This necessitates abandoning even the most basic form of empiricism by “rejecting the evidence of one’s own eyes and ears”, to paraphrase Orwell.[5] The fanatic is thus essentially incorrigible and has no epistemic or moral uncertainty, even in the face of widespread opposition (Gollwitzer et al., 2022).[6]
This is the definition of ideological fanatic as far as I can tell! You are telling me 200-250 million Christians worldwide fit this description?
This is the definition of ideological fanatic as far as I can tell! You are telling me 200-250 million Christians worldwide fit this description?
I agree that’s too high, not on that strict description of fanaticism. But David also writes “For brevity, we focus here on support for ideological violence as the best proxy for ideological fanaticism.” So you may be right that there’s a bit of motte-and-bailey going on with who gets counted as “fanatical”. But I think the post is clear about what it is or isn’t saying.
And just to be clear, I think the true numbers would probably be somewhat shocking even for the strict/extreme definition. Like maybe a quarter of those 200-250 million, in my estimate. Africa still has witch burnings, some places in the US are very religious and almost every family there has this one (extended-)family member who is really fanatical, and I was mostly thinking about US big cities just now but in rural populations it’s probably even more pronounced.
But I think the post is clear about what it is or isn’t saying.
I mean, I do think the post is clear, and the post is saying “tens of millions of people in the world consider any doubt or deviation from these dogmas as not only wrong but evil, culminating in a total “soldier mindset” which defends the pre-existing ideology at all costs. This necessitates abandoning even the most basic form of empiricism by “rejecting the evidence of one’s own eyes and ears”, to paraphrase Orwell. These people are thus essentially incorrigible and have no epistemic or moral uncertainty, even in the face of widespread opposition”
I agree the post is clear on that, but it also extremely unlikely to me.
And you seem to agree! The right number here is not 50-75 million people! Are you telling me that 50-75 million people have “abandoned even the most basic form of empiricism”, “defending their pre-existing ideology at all costs”?
This just doesn’t match any historical conflicts or ideological social movements. Yes, group dynamics can drive people into doing weird and aggressive things, but these are descriptions of what individual people would do, and that a substantial fraction of them would be incapable of reform. In contrast to that, in the moment those group dynamics end, the vast majority of people caught up in these things turn out to be normal and well-adjusted, with the above being a terrible description.
almost every family there has this one family member who is really fanatical
I don’t think I’ve ever met someone who fits the description above. One of the key lines is that “they are incorrigible and have no epistemic or moral uncertainty, even in the face of widespread opposition”. But that doesn’t match what usually goes on here at all. The vast majority of people behaving fanatically would stop doing so when facing widespread opposition, and this is certainly true of the family members I’ve seen here. People are religious fundamentalists because the people around them are. If the people around them stop and start pushing back, a very very small fraction of people would end up insisting for the rest of their lives on their previous beliefs.
The language generally does sound a little strong, but I’d guess it to be directionally correct and that your points wouldn’t significantly change the post’s conclusions (though I admit I’ve only skimmed the post). Like, it’s true that a lot of people will change their minds if the social context changes, but if the ideology manages to maintain a stable-enough social context or one that shifts adaptively enough, then those people’s attitudes can stay quite resilient.
And even if huge numbers of people did change their minds, it’s possible for some not to, e.g. because their psychology for one reason or another ends up leaving them no line of retreat, so that anything ends up being less painful than changing one’s mind. The post also notes that it may be enough for a pretty small number of people to be fanatics, if those people end up in control of a state.
Generally I think that Duncan’s heuristic of betting on existence is a pretty good one that’s generally correct, and also applies for the case of “extreme fanatics do exist”.
I don’t know how I would differentiate paragraphs like the above from ravings of religious people talking about demons or devils.
It seems like in these posts all negative attributes must be assigned to these people, and all their cognitive leanings must be absolute and unwielding.
This seems like a strawman? The post was assigning some negative attributes to these people, not all of them. For one, extreme tribalism implies a loyalty to your own tribe, which is generally seen as a positive virtue. (Stereotypical demons don’t even have that quality.)
How to differentiate this from talk about demons or devils—well, most obviously, demons and devils are supernatural and incompatible with any naturalistic understanding of the world. Some variables within the brain getting stuck in an extreme setting is not. (I do find it a plausible claim that e.g. a superintelligence capable of arbitrarily manipulating such a person’s environment could always find some way of getting the person to change their mind, but I think the post is most reasonably read as “incapable of changing their minds for most practical purposes”.)
The language generally does sound a little strong, but I’d guess it to be directionally correct and that your points wouldn’t significantly change the post’s conclusions (though I admit I’ve only skimmed the post).
[...]
Generally I think that Duncan’s heuristic of betting on existence is a pretty good one that’s generally correct, and also applies for the case of “extreme fanatics do exist”.
I am not arguing against “there are literally no people who could meaningfully be described as having these attributes”, but the post goes on to classify over 500 million people worldwide as having these attributes!
My guess is you would agree with me here if you read the article beyond skimming.
In conclusion, accounting for potential overlap between categories, perhaps 500 million to 1 billion people, roughly 6-12% of the world population, may plausibly be classified as ideological fanatics
Again, their only definition of an “ideological fanatic” is that extremely extremely strong summary I linked above. There is no section of the post that’s like “of course, the vast vast majority of ideological fanatics do not think anything like this and are not well-described by this, and are largely behaving this way due to social momentum, and are maybe mildly on a spectrum in this direction”. It just creates an extremely intense boogeyman of “ideological fanatics” then classifies ~10% of the world population as matching that description.
their only definition of an “ideological fanatic” is that extremely extremely strong summary I linked above. There is no section of the post that’s like “of course, the vast vast majority of ideological fanatics do not think anything like this and are not well-described by this, and are largely behaving this way due to social momentum, and are maybe mildly on a spectrum in this direction”
I think these excerpts are saying something like that?
One overarching characteristic of the fanatical worldview is black-and-white thinking (good vs. evil, us vs. them) with no room for nuance. Let’s not make the same mistake. Like most phenomena, ideological fanaticism exists on a continuum. Furthest from fanaticism are those enlightened few who, following reason and evidence, act with benevolence towards all. A vast middle ground is occupied by religious traditionalists, hyper-partisan activists, conspiracy theorists, and many others. Indeed, a mild form of ideological fanaticism is arguably human nature: we are all somewhat prone to overconfidence, motivated reasoning, and tribalistic in-group favoritism and outgroup discrimination (e.g., Kunda, 1990; Diehl, 1990; Hewstone et al., 2002). [...]
Ideological fanaticism is not just a single sliding scale. Rather, it is multidimensional, that is, people can exhibit different levels of each fanatical triad component. The most dangerous form of ideological fanaticism requires elevated levels of all three characteristics. A hypothetical ‘Bayesian Nazi’, for instance, would lack absolute certainty and thus remain open to changing his mind. Similarly, without Manichean hatred, there is no motivation for mass harm, and without a willingness to use violence, even the most hateful beliefs remain inert.
Nor are fanatical movements monolithic.[13] While their leaders often were malignant narcissists, their followers are frequently ordinary people desperately seeking meaning and certainty in a chaotic, disappointing world (Hoffer, 1951; Kruglanski et al., 2014; Tietjen, 2023). Not all are true believers, either: some merely conform to group pressure, others are cynical opportunists, and many fall somewhere in between.[14] Many fanatics are capable of eventual reform, so we should not demonize them as irredeemably evil.
I agree these talk about potential scales, but it seems to me that they describe “ideological fanatics” as a pretty extreme point on that scale, and then classify at least hundreds of millions of people as falling on that side of the scale?
Not all are true believers, either: some merely conform to group pressure, others are cynical opportunists, and many fall somewhere in between.[14] Many fanatics are capable of eventual reform, so we should not demonize them as irredeemably evil.
These are not the right quantitative adjectives here. The correct ones to use would be:
Practically no people we classify as “ideological fanatics” are true believers
Virtually all (of the 500M+) fanatics are capable of eventual reform
Like, I don’t see how you could use the original quantifiers (“not all” and “many”), and after multiplying them through with the quantitative estimates not arrive at numbers at least 3 OOMs too high for these traits.
but it seems to me that they describe “ideological fanatics” as a pretty extreme point on that scale
I agree that the original text is ambiguous in this regard and that there are reasonable grounds for your reading. Personally I interpret sentences like
ideological fanaticism exists on a continuum
The most dangerous form of ideological fanaticism requires elevated levels of all three characteristics
their followers are frequently ordinary people
to mean something like “we are giving descriptions for the most extreme version of ideological fanaticism as that’s the easiest to gesture toward, but will also include people with less extreme versions when trying to estimate the sizes of the movements”.
I think it’s also relevant that the section on dogmatic certainty that you quoted beings with
The most ardent fanatics
implying that not all fanatics are this extreme; and of course the section on dogmatic certainty was one of the three sub-dimensions for ideological fanaticism rather than the overall definition.
But I think “which one of these readings is more correct” gets pretty subjective and impossible to resolve, so for me the more important test is something like… “if the authors could be read as making either a strong claim or a weaker one, how much do their conclusions depend on the stronger claim?”.
And it seems to me that on an interpretation where they only mean something like “virtually all fanatics are capable of eventual reform in principle, but in practice may be stuck in an environment where that is very unlikely”… then the various dangers that they outline, like “Ideological fanaticism increases the risk of war and conflict” or “Fanatical retributivism may lead to astronomical suffering”, still sound plausible.
Of course, it’s still fair to criticize the authors for e.g. being unclear about this or for implying a stronger claim when a weaker claim would suffice, but I wouldn’t strong-downvote them for that.
With regard to the bit about the 200-250 million Christians, that section does contain this paragraph
How many ideological fanatics are out there? Formulating a precise estimate is nearly impossible, as fanaticism exists on a multidimensional continuum with no clear demarcations, and because good data is sparse. Therefore, the numbers below are merely rough approximations based on limited research. For brevity, we focus here on support for ideological violence as the best proxy for ideological fanaticism. Endorsing ideological violence usually presupposes dogmatism and tribalistic hatred, since one needs to confidently believe the hated target group is deserving of punishment in order to justify violence. Another limitation is that we mostly rely on survey data[45], not actual behavior; this may overestimate fanaticism (if claimed support for violence is mere “cheap talk”) or underestimate it (“social desirability bias”).
I think a reasonable reading of this section/paragraph is also something like “for purposes of this section, we are defining an ideological fanatic as having some combination of these three traits that’s high enough for them to endorse ideological violence” [not implying that they would necessarily all be maxed out on the “dogmatic certainty” dimension].
Here we could apply a similar test of… “if we interpret the 200-250 million Christians narrowly as only being people who endorse ideological violence, rather than assuming that they’re necessarily incapable of changing their minds, does this still e.g. increase the risk of war and conflict?”. I think the answer is pretty clearly yes.
I have strong-downvoted both of those posts! I think they do a mixture of the “taking propaganda at face value” thing, seem to largely be in some kind of negative affect spiral around imagining arbitrarily bad qualities they can assign to people, seem to take an enormously naive view of the psychopathology literature, seem to consistently avoid dealing with the moral reflection dynamics I outline in this post, and seem very rooted in negative utilitarianism, which I disagree with[1].
Overall, I am pretty sad about that whole set of stuff. I somewhat wish I could talk more to people about it, but the negative utilitarianism-leaning stuff usually makes good-faith discourse a lot harder (since the bottom-line for my interlocutors seems clearly written).
Like man, those posts are so frustrating to read. They just… assert things. No epistemic statuses, no caveats, just stuff like this:
It honestly reads to me kind of like true crime podcasts? They love always describing everyone they cover with these kinds of extreme characterizations which basically never hold up when you look into them. My honest best guess is these posts serve a kind of similar role within our community.
I am confident the psychological characterization in paragraphs like the above is wrong. “Defends pre-existing ideology at all costs”. Come on, no, of course not “all costs”. Indeed a huge fraction of people caught up in these ideologies end up deconverting or drastically mellowing their views when their social context changes. These seem hilariously strawmanny absolutes that get ascribed to people here.
This is exactly the kind of thing I mean when I say people seem really got by propaganda, and when people engage in the fundamental attribution error. Like, I don’t know how I would differentiate paragraphs like the above from ravings of religious people talking about demons or devils.
It seems like in these posts all negative attributes must be assigned to these people, and all their cognitive leanings must be absolute and unwielding. They must “defend the pre-existing ideology at all costs”, they are “utterly convinced”, they are “textbook dogmatists”, they place “absolute faith” in the holy texts. Come on, this is not how people work.
and furthermore I think is evidence that something pretty deep is going wrong in how they are thinking about things
Thanks for sharing that take, which I find largely quite bizarre and surprising. I continue to think those posts are super valuable.
I can understand finding the negative(-leaning) utilitarianism codedness of the writings annoying, but I don’t see why you think it makes good-faith discussion a lot harder. From my perspective, a lot of writings on LW are “yay, hurrah team human”-coded in a way that annoys the crap out of me and makes me want to punch things, but it’s not like that means I can’t get valuable things out of the writings or have to treat the posters here as necessarily adversarial.
The subject of the sentence you put in quotation marks was “The fanatic”—as in, “the archetypal example of the fanatic.” This sounds to me like more of a writing style issue. Like, it wouldn’t even occur to me to assume that the post is saying that every believer of a harmful ideology is like that. More that there’s a fanaticism attractor and that people at the center of it really do approach that described extreme—which I think is true? I guess that’s the thing you’re contesting, but I feel like history contains some examples of atrocities that are hard to explain without choosing at least one of the following: either some people can get incredibly fanatical, or some people are sadistic/evil and may use fanaticism/ideology as a cover. Either way, at least one of the posts’ messages must be true?
There are some far-future relevance speculations where my model of what is likely to happen with AI is different from David’s, so I think it’s just much more likely that AIs will take over and it won’t matter what the humans who built it were like. But that doesn’t really invalidate too much—I mean part of the posts are also about what sort of qualities we wouldn’t want to see in AIs that we build. On your point about how AI-aided moral reflection and overcoming resource scarcity would reduce fanaticism or other bad consequences from “bad values,” I think that sort of point is underappreciated in some EA circles, but it’s not like it’s obvious that this is what’s going to happen. I think the posts we’re discussing engage well with reasons why reflection might not solve all the problems.
The rest of the post then tries to argue that these kinds of mental traits are relatively widespread, and argues from that incorrigibility and absolute fanaticism.
I agree an “archetype” definition could be fine. But it’s IMO clearly not what’s going on. The post makes no attempt at clarifying how far of an outlier all of the above descriptions are, and later includes sections like:
This is obviously absurd! 200-250 million Christians absolutely do not fit the description that I quoted in my comment above, which is the only definition they give for what they mean by “ideological fanatics”:
This is obviously not how this works. This is such a blatant example of category gerrymandering and the noncentral fallacy that I think my upset is very justified.
This is the definition of ideological fanatic as far as I can tell! You are telling me 200-250 million Christians worldwide fit this description?
I agree that’s too high, not on that strict description of fanaticism. But David also writes “For brevity, we focus here on support for ideological violence as the best proxy for ideological fanaticism.” So you may be right that there’s a bit of motte-and-bailey going on with who gets counted as “fanatical”. But I think the post is clear about what it is or isn’t saying.
And just to be clear, I think the true numbers would probably be somewhat shocking even for the strict/extreme definition. Like maybe a quarter of those 200-250 million, in my estimate. Africa still has witch burnings, some places in the US are very religious and almost every family there has this one (extended-)family member who is really fanatical, and I was mostly thinking about US big cities just now but in rural populations it’s probably even more pronounced.
I mean, I do think the post is clear, and the post is saying “tens of millions of people in the world consider any doubt or deviation from these dogmas as not only wrong but evil, culminating in a total “soldier mindset” which defends the pre-existing ideology at all costs. This necessitates abandoning even the most basic form of empiricism by “rejecting the evidence of one’s own eyes and ears”, to paraphrase Orwell. These people are thus essentially incorrigible and have no epistemic or moral uncertainty, even in the face of widespread opposition”
I agree the post is clear on that, but it also extremely unlikely to me.
And you seem to agree! The right number here is not 50-75 million people! Are you telling me that 50-75 million people have “abandoned even the most basic form of empiricism”, “defending their pre-existing ideology at all costs”?
This just doesn’t match any historical conflicts or ideological social movements. Yes, group dynamics can drive people into doing weird and aggressive things, but these are descriptions of what individual people would do, and that a substantial fraction of them would be incapable of reform. In contrast to that, in the moment those group dynamics end, the vast majority of people caught up in these things turn out to be normal and well-adjusted, with the above being a terrible description.
I don’t think I’ve ever met someone who fits the description above. One of the key lines is that “they are incorrigible and have no epistemic or moral uncertainty, even in the face of widespread opposition”. But that doesn’t match what usually goes on here at all. The vast majority of people behaving fanatically would stop doing so when facing widespread opposition, and this is certainly true of the family members I’ve seen here. People are religious fundamentalists because the people around them are. If the people around them stop and start pushing back, a very very small fraction of people would end up insisting for the rest of their lives on their previous beliefs.
I have met Christians like that and I don’t see why those numbers would be too high. We live in extreme filter bubbles.
The language generally does sound a little strong, but I’d guess it to be directionally correct and that your points wouldn’t significantly change the post’s conclusions (though I admit I’ve only skimmed the post). Like, it’s true that a lot of people will change their minds if the social context changes, but if the ideology manages to maintain a stable-enough social context or one that shifts adaptively enough, then those people’s attitudes can stay quite resilient.
And even if huge numbers of people did change their minds, it’s possible for some not to, e.g. because their psychology for one reason or another ends up leaving them no line of retreat, so that anything ends up being less painful than changing one’s mind. The post also notes that it may be enough for a pretty small number of people to be fanatics, if those people end up in control of a state.
Generally I think that Duncan’s heuristic of betting on existence is a pretty good one that’s generally correct, and also applies for the case of “extreme fanatics do exist”.
This seems like a strawman? The post was assigning some negative attributes to these people, not all of them. For one, extreme tribalism implies a loyalty to your own tribe, which is generally seen as a positive virtue. (Stereotypical demons don’t even have that quality.)
How to differentiate this from talk about demons or devils—well, most obviously, demons and devils are supernatural and incompatible with any naturalistic understanding of the world. Some variables within the brain getting stuck in an extreme setting is not. (I do find it a plausible claim that e.g. a superintelligence capable of arbitrarily manipulating such a person’s environment could always find some way of getting the person to change their mind, but I think the post is most reasonably read as “incapable of changing their minds for most practical purposes”.)
[...]
I am not arguing against “there are literally no people who could meaningfully be described as having these attributes”, but the post goes on to classify over 500 million people worldwide as having these attributes!
My guess is you would agree with me here if you read the article beyond skimming.
Again, their only definition of an “ideological fanatic” is that extremely extremely strong summary I linked above. There is no section of the post that’s like “of course, the vast vast majority of ideological fanatics do not think anything like this and are not well-described by this, and are largely behaving this way due to social momentum, and are maybe mildly on a spectrum in this direction”. It just creates an extremely intense boogeyman of “ideological fanatics” then classifies ~10% of the world population as matching that description.
I think these excerpts are saying something like that?
I agree these talk about potential scales, but it seems to me that they describe “ideological fanatics” as a pretty extreme point on that scale, and then classify at least hundreds of millions of people as falling on that side of the scale?
These are not the right quantitative adjectives here. The correct ones to use would be:
Practically no people we classify as “ideological fanatics” are true believers
Virtually all (of the 500M+) fanatics are capable of eventual reform
Like, I don’t see how you could use the original quantifiers (“not all” and “many”), and after multiplying them through with the quantitative estimates not arrive at numbers at least 3 OOMs too high for these traits.
I agree that the original text is ambiguous in this regard and that there are reasonable grounds for your reading. Personally I interpret sentences like
to mean something like “we are giving descriptions for the most extreme version of ideological fanaticism as that’s the easiest to gesture toward, but will also include people with less extreme versions when trying to estimate the sizes of the movements”.
I think it’s also relevant that the section on dogmatic certainty that you quoted beings with
implying that not all fanatics are this extreme; and of course the section on dogmatic certainty was one of the three sub-dimensions for ideological fanaticism rather than the overall definition.
But I think “which one of these readings is more correct” gets pretty subjective and impossible to resolve, so for me the more important test is something like… “if the authors could be read as making either a strong claim or a weaker one, how much do their conclusions depend on the stronger claim?”.
And it seems to me that on an interpretation where they only mean something like “virtually all fanatics are capable of eventual reform in principle, but in practice may be stuck in an environment where that is very unlikely”… then the various dangers that they outline, like “Ideological fanaticism increases the risk of war and conflict” or “Fanatical retributivism may lead to astronomical suffering”, still sound plausible.
Of course, it’s still fair to criticize the authors for e.g. being unclear about this or for implying a stronger claim when a weaker claim would suffice, but I wouldn’t strong-downvote them for that.
With regard to the bit about the 200-250 million Christians, that section does contain this paragraph
I think a reasonable reading of this section/paragraph is also something like “for purposes of this section, we are defining an ideological fanatic as having some combination of these three traits that’s high enough for them to endorse ideological violence” [not implying that they would necessarily all be maxed out on the “dogmatic certainty” dimension].
Here we could apply a similar test of… “if we interpret the 200-250 million Christians narrowly as only being people who endorse ideological violence, rather than assuming that they’re necessarily incapable of changing their minds, does this still e.g. increase the risk of war and conflict?”. I think the answer is pretty clearly yes.