But somehow, humans have the habit of taking good and reasonable ideas, making strawman versions of them, and presenting them as the true thing. (I suppose this is about signalling—the more extreme version of X you believe, the more respected you are in the crowd that chose X as an applause light, even if that version does not make sense anymore. It’s no longer about making sense, but about showing loyalty.)
I think this is the root of all a lot of evil. To add to this, I’m not sure about the exact game theoretic mechanism, but signaling loyalty seems really important for human cooperation, so we can’t just tell people to stop doing this. (To the extent that some people do stop using ideology for loyalty signaling upon learning about it or for other reasons, it seems to hurt their ability for collective action.) To me this is one of the biggest open problems in human group rationality.
The real-world solution seems to be leaders who preach the extreme version of the idea, but their actual actions are much more moderate. So the extremism of applause lights is balanced by the difference between what is promised and what gets actually done.
(This of course opens another bag of problems, for example it is not guaranteed that the leader will omit the stupid parts and keep the reasonable parts; it could easily be the other way round. But increasing control over the leaders would restore the original problem...)
For example, you could have a private school with a director who publicly preaches extreme constructivism, but in fact the school would use the usual repetition and memorization, only put a greater emphasis on understanding first. Most of the true believers would be happy to hear what they want to hear, and would not pay attention to details. Most non-believers would be placated with actual good outcomes. And the few ones who… -- well, you can’t make everyone happy anyway.
Problem is, this solution is difficult to scale. If someone encouraged by the success would want to create a similar school in their city, they would probably go too far. And even the original director would have to carefully explain to their teachers what exactly is expected of them. (The true believers would try to shift the result in one direction, the lukewarm ones in another.) At last, the director’s successor would ruin the balance, and the school would either become a crazy one, or an average one.
When I stop to think of people I support who I would peg as “extreme in words, moderate in actions”, I think I feel a sense of overall safety that might be relevant here.
Let’s say I’m in a fierce, conquering mood. I can put my weight behind their extremism, and feel powerful. I’m Making A Difference, going forth and reshaping the world a little closer to utopia.
When I’m in a defeatist mood, where nothing makes sense and I feel utterly hopeless, I can *also* get behind the extremism—but it’s in a different light, now. It’s more, “I am so small, and the world is so big, but I can still live by what I feel is right”.
Those are really emotionally powerful and salient times for me, and ones that have a profound effect on my sense of loyalty to certain causes. But most of the time, I’m puttering along and happy to be in the world of moderation. Intellectually, I understand that moderation is almost always going to be the best way forward; emotionally, it’s another story entirely.
Upon first reading, I had the thought that a lot of people don’t notice the extreme/moderate dichotomy of most of their leaders. I still think that’s true. And then a lot of people do learn of that dichotomy, and they become disgusted by it, and turn away from anyone who falls in that camp. Which makes sense, honesty is a great virtue, why can’t they just say what they mean? But then I look at myself, and while it doesn’t feel *optimal* to me, it does feel like just another element of playing the game of power. There’s this skill of reading between the lines that I think most people know is there, but they’re a little reluctant to look straight at it.
I think this is the root of
alla lot of evil. To add to this, I’m not sure about the exact game theoretic mechanism, but signaling loyalty seems really important for human cooperation, so we can’t just tell people to stop doing this. (To the extent that some people do stop using ideology for loyalty signaling upon learning about it or for other reasons, it seems to hurt their ability for collective action.) To me this is one of the biggest open problems in human group rationality.The real-world solution seems to be leaders who preach the extreme version of the idea, but their actual actions are much more moderate. So the extremism of applause lights is balanced by the difference between what is promised and what gets actually done.
(This of course opens another bag of problems, for example it is not guaranteed that the leader will omit the stupid parts and keep the reasonable parts; it could easily be the other way round. But increasing control over the leaders would restore the original problem...)
For example, you could have a private school with a director who publicly preaches extreme constructivism, but in fact the school would use the usual repetition and memorization, only put a greater emphasis on understanding first. Most of the true believers would be happy to hear what they want to hear, and would not pay attention to details. Most non-believers would be placated with actual good outcomes. And the few ones who… -- well, you can’t make everyone happy anyway.
Problem is, this solution is difficult to scale. If someone encouraged by the success would want to create a similar school in their city, they would probably go too far. And even the original director would have to carefully explain to their teachers what exactly is expected of them. (The true believers would try to shift the result in one direction, the lukewarm ones in another.) At last, the director’s successor would ruin the balance, and the school would either become a crazy one, or an average one.
When I stop to think of people I support who I would peg as “extreme in words, moderate in actions”, I think I feel a sense of overall safety that might be relevant here.
Let’s say I’m in a fierce, conquering mood. I can put my weight behind their extremism, and feel powerful. I’m Making A Difference, going forth and reshaping the world a little closer to utopia.
When I’m in a defeatist mood, where nothing makes sense and I feel utterly hopeless, I can *also* get behind the extremism—but it’s in a different light, now. It’s more, “I am so small, and the world is so big, but I can still live by what I feel is right”.
Those are really emotionally powerful and salient times for me, and ones that have a profound effect on my sense of loyalty to certain causes. But most of the time, I’m puttering along and happy to be in the world of moderation. Intellectually, I understand that moderation is almost always going to be the best way forward; emotionally, it’s another story entirely.
Upon first reading, I had the thought that a lot of people don’t notice the extreme/moderate dichotomy of most of their leaders. I still think that’s true. And then a lot of people do learn of that dichotomy, and they become disgusted by it, and turn away from anyone who falls in that camp. Which makes sense, honesty is a great virtue, why can’t they just say what they mean? But then I look at myself, and while it doesn’t feel *optimal* to me, it does feel like just another element of playing the game of power. There’s this skill of reading between the lines that I think most people know is there, but they’re a little reluctant to look straight at it.