Your overall point seems true and important. The virtues that are enshrined by the rat-fic tradition are primarily “sanity in the midst of insanity (eg. social decoupling and thinking for yourself)” and “heroic responsibility”. These attitudes are not strictly counter to cooperation or working with the existing systems of the world, but they sure do tend to push in the opposite direction. The more you distrust the world, the less you think that you should cooperate with it.
And further, actual practice, “thinking for yourself” directly cuts against human political coordination, since political coalitions almost always maintain unity by coordinating their beliefs, and challenging those coordination beliefs undermines the coalition.
(I find it amusing when people sometimes say that the “one job” of rationalists is to be able to coordinate. I think that’s ass-backwards. The rationalist tradition is about prioritizing independence of thought and the epistemology to discern the truth at the expense of human coordination. There’s some hope there’s another even stronger mode of coordination on the other side of the uncanny valley, but that, so far, remains an unvalidated hope.[1])
However, with regards to this specific point...
But the worlds didn’t have to be that bad! The writers chose to design them that way—I expect in significant part because that provides a narratively compelling backdrop for the thing they wanted to write about, which was their heroes taking over the world.
Eliezer has said that one of the reasons he writes fanfiction, is that he doesn’t have to invent the world. All of the horror and badness was already present in the source material.
Azkaban, as Rowling depicted it in the wizard world, is entirely realistic. If there are no Dementors in American prisons, it’s because American politicians have no Dementors to use, not because they’re better people than the Wizengamot. Sexual assault is routine in American prisons and that could easily be prevented with video cameras. American prisons are worse than Azkaban in ways that Rowling couldn’t easily have imitated without breaking her readers’ suspension of disbelief. At least the wizarding world isn’t imprisoning marijuana users that we ever saw.
Even so, if Azkaban were in a world of my own invention, someone might question the realism of Harry’s reaction to Azkaban, versus other people in magical Britain seeming not to notice Azkaban as a moral horror. (Just like Americans don’t notice the moral horror! Rowling was not being unrealistic!) How is it that Harry sees all these utilons that can be picked up by ending Azkaban, that nobody else has seen? (Answer: it is not possible for any arbitrary economic actor to make a hundred thousand Galleons of profit if they have the insight that Azkaban is needlessly cruel, so standard economics does not predict moral efficiency the way it predicts efficient stock markets.) Perhaps Eliezer Yudkowsky only invented Azkaban to be triumphed over by his allegedly superior hero, and put it into his world as a straw inefficiency, a weakman…
But I didn’t invent Azkaban, it’s right there in canon, and millions of readers read J. K. Rowling’s stories and (correctly) accepted this as a routine background premise rather than claiming (incorrectly) that no (flawed) democracy (the size of a small town) would ever do such a thing and that she was just putting Azkaban there to show off her hero’s moral superiority.
Indeed, a hope that I personally share. But think the folks who are inclined to say that coordination is the “one job”, or similar, of rationalists are missing the plot. Coordinating groups larger than 50 people to accomplish a goal or a political change, by way of accurate views instead of adaptive stories and taboo beliefs, is an unsolved challenge.
I’ll note that overall, the EAs seem to be doing somewhat better than the rationalists at working in groups to get stuff done, and also, they are relatively less free-thinking, and have more taboo beliefs.
Eliezer has said that one of the reasons he writes fanfiction, is that he doesn’t have to invent the world. All of the horror and badness was already present in the source material.
A large majority of fantasy settings don’t have literal hells as a key component, so I think my point is still applicable to Project Lawful if you replace “design them that way” with the more general “select for that trait”.
I do agree that this is a good point with regards to HPMoR, which is one reason why I didn’t include HPMoR in my original list of examples.
I think one of the other works has a wrong explanation as well, namely
(title)
Worth The Candle
(heavy spoilers)
WTC actually has an extreme amount of power-seeking behavior, exactly as the post says, but the actual end happens that way, it appears, because the author believes that retiring his characters in a manner of their choosing is desirable. To this end, one character abdicates responsibility, living out their life on earth. Another is given control of the universe, abolishes Hell, and then allows all the universe’s participants to weigh in on their preferences, with the new god character creating a series of heavens, several of which they find distasteful. I… am not sure how much gentler the author could have made this, to be honest. A singleton forms and allows diverse values— basically ideal.
What I mean is that your theory pays out as a ban on stories about any singleton, including in stories where the singleton does un-singletonly things. If that’s the case, any author who expects there are RSI-dynamics in their story is verboten, which is rough if it makes sense!
My understanding of your dynamic has a high false-positive rate: if I wrote fiction about Napoleon, it would look power-seeking (because he was), but this doesn’t seem like a useful red flag to me.
Your overall point seems true and important. The virtues that are enshrined by the rat-fic tradition are primarily “sanity in the midst of insanity (eg. social decoupling and thinking for yourself)” and “heroic responsibility”. These attitudes are not strictly counter to cooperation or working with the existing systems of the world, but they sure do tend to push in the opposite direction. The more you distrust the world, the less you think that you should cooperate with it.
And further, actual practice, “thinking for yourself” directly cuts against human political coordination, since political coalitions almost always maintain unity by coordinating their beliefs, and challenging those coordination beliefs undermines the coalition.
(I find it amusing when people sometimes say that the “one job” of rationalists is to be able to coordinate. I think that’s ass-backwards. The rationalist tradition is about prioritizing independence of thought and the epistemology to discern the truth at the expense of human coordination. There’s some hope there’s another even stronger mode of coordination on the other side of the uncanny valley, but that, so far, remains an unvalidated hope.[1])
However, with regards to this specific point...
Eliezer has said that one of the reasons he writes fanfiction, is that he doesn’t have to invent the world. All of the horror and badness was already present in the source material.
Indeed, a hope that I personally share. But think the folks who are inclined to say that coordination is the “one job”, or similar, of rationalists are missing the plot. Coordinating groups larger than 50 people to accomplish a goal or a political change, by way of accurate views instead of adaptive stories and taboo beliefs, is an unsolved challenge.
I’ll note that overall, the EAs seem to be doing somewhat better than the rationalists at working in groups to get stuff done, and also, they are relatively less free-thinking, and have more taboo beliefs.
A large majority of fantasy settings don’t have literal hells as a key component, so I think my point is still applicable to Project Lawful if you replace “design them that way” with the more general “select for that trait”.
I do agree that this is a good point with regards to HPMoR, which is one reason why I didn’t include HPMoR in my original list of examples.
I think one of the other works has a wrong explanation as well, namely
(title)
Worth The Candle
(heavy spoilers)
WTC actually has an extreme amount of power-seeking behavior, exactly as the post says, but the actual end happens that way, it appears, because the author believes that retiring his characters in a manner of their choosing is desirable. To this end, one character abdicates responsibility, living out their life on earth. Another is given control of the universe, abolishes Hell, and then allows all the universe’s participants to weigh in on their preferences, with the new god character creating a series of heavens, several of which they find distasteful. I… am not sure how much gentler the author could have made this, to be honest. A singleton forms and allows diverse values— basically ideal.
This is precisely the attitude I am critiquing, and therefore I don’t find your comment very persuasive.
What I mean is that your theory pays out as a ban on stories about any singleton, including in stories where the singleton does un-singletonly things. If that’s the case, any author who expects there are RSI-dynamics in their story is verboten, which is rough if it makes sense!
My understanding of your dynamic has a high false-positive rate: if I wrote fiction about Napoleon, it would look power-seeking (because he was), but this doesn’t seem like a useful red flag to me.