(Self-review.) I think this post was underappreciated. At the time, I didn’t want to emphasize the social–historical angle because it seemed like too much of a distraction from the substantive object-level point, but I think this post is pointing at a critical failure in how the so-called “rationalist” movement has developed over time.
At the end of the post, I quote Steven Kaas writing in 2008: “if you’re interested in producing truth, you will fix your opponents’ arguments for them.” I see this kind of insight as at the core of what made the Sequences so valuable: a clear articulation of how a monomaniacal focus on the truth implies counterintuitive social behavior. Normatively, it shouldn’t be unusual for people to volunteer novel arguments that support their interlocutor’s belief—that’s just something you’d do naturally in the course of trying to figure out the right answer—but it is unusual, because most disagreements are actually disguised conflicts.
And yet less than a decade later (as documented byRob Bensinger in the post that this post responds to), we see Eliezer Yudkowsky proclaiming that “Eliezer and Holden are both on record as saying that ‘steelmanning’ people is bad and you should stop doing it”—a complete inversion of Kaas’s advice! (Kaas didn’t use the specific jargon term “steelmanning”, but that’s obviously inessential.)
For clarity, I want to recap that one more time in fewer words, to distill the essence of the inversion—
In 2008, the community wisdom was that fixing your interlocutor’s arguments for them (what was not yet called “steelmanning”) was a good thing. The warrant cited for this advice was that it’s something you do “if you’re interested in producing truth”.
In 2017, the community wisdom was that fixing your interlocutor’s arguments for them (by then known as “steelmanning”) was “bad and you should stop doing it” (!!). The warrant cited for this advice was that “Eliezer and Holden” (who?) “are both on record as saying” it.
Why? What changed? How could something that was considered obviously good in 2008, be considered bad in 2017? Did no one else notice? Are we not supposed to notice? I have my own tentative theories, but I’m interested in what Raymond Arnold and Ruby Bloom think (relavant to the topic of “[keeping] alive the OG vision of improving human rationality”).
fwiw I think there is a good thing about steelmanning and a different good thing about ITT passing. (Which seems plausibly consistent with Rob’s title ITT-passing and civility are good; “charity” is bad; steelmanning is niche, and also your post title here. I haven’t reread either yet but am responding since I was tagged)
ITT passing is good for making sure you are having a conversation that changes people’s minds, and not getting confused/mislead about what other people believe.
Steelmanning is good for identifying the strongest forms of arguments in a vacuum, which is useful for exploring the argument space but also prone to spending time on something that nobody believes or cares about, which is sometimes worth it and sometimes not. (it also often is part of a process that misleads people about what a person or group believes)
Which of those is more important most of the time? I dunno, the answer is AFAICT “each consideration is important enough to track that you should pay attention to them periodically.” And it feels like attempts to pin this down further feel more like some kind of culture war that isn’t primarily about the object-level fact of how often they are useful.
(apologies if I have missed a major point here, replying quickly at a busy time)
(Self-review.) I think this post was underappreciated. At the time, I didn’t want to emphasize the social–historical angle because it seemed like too much of a distraction from the substantive object-level point, but I think this post is pointing at a critical failure in how the so-called “rationalist” movement has developed over time.
At the end of the post, I quote Steven Kaas writing in 2008: “if you’re interested in producing truth, you will fix your opponents’ arguments for them.” I see this kind of insight as at the core of what made the Sequences so valuable: a clear articulation of how a monomaniacal focus on the truth implies counterintuitive social behavior. Normatively, it shouldn’t be unusual for people to volunteer novel arguments that support their interlocutor’s belief—that’s just something you’d do naturally in the course of trying to figure out the right answer—but it is unusual, because most disagreements are actually disguised conflicts.
And yet less than a decade later (as documented by Rob Bensinger in the post that this post responds to), we see Eliezer Yudkowsky proclaiming that “Eliezer and Holden are both on record as saying that ‘steelmanning’ people is bad and you should stop doing it”—a complete inversion of Kaas’s advice! (Kaas didn’t use the specific jargon term “steelmanning”, but that’s obviously inessential.)
For clarity, I want to recap that one more time in fewer words, to distill the essence of the inversion—
In 2008, the community wisdom was that fixing your interlocutor’s arguments for them (what was not yet called “steelmanning”) was a good thing. The warrant cited for this advice was that it’s something you do “if you’re interested in producing truth”.
In 2017, the community wisdom was that fixing your interlocutor’s arguments for them (by then known as “steelmanning”) was “bad and you should stop doing it” (!!). The warrant cited for this advice was that “Eliezer and Holden” (who?) “are both on record as saying” it.
Why? What changed? How could something that was considered obviously good in 2008, be considered bad in 2017? Did no one else notice? Are we not supposed to notice? I have my own tentative theories, but I’m interested in what Raymond Arnold and Ruby Bloom think (relavant to the topic of “[keeping] alive the OG vision of improving human rationality”).
fwiw I think there is a good thing about steelmanning and a different good thing about ITT passing. (Which seems plausibly consistent with Rob’s title ITT-passing and civility are good; “charity” is bad; steelmanning is niche, and also your post title here. I haven’t reread either yet but am responding since I was tagged)
ITT passing is good for making sure you are having a conversation that changes people’s minds, and not getting confused/mislead about what other people believe.
Steelmanning is good for identifying the strongest forms of arguments in a vacuum, which is useful for exploring the argument space but also prone to spending time on something that nobody believes or cares about, which is sometimes worth it and sometimes not. (it also often is part of a process that misleads people about what a person or group believes)
Which of those is more important most of the time? I dunno, the answer is AFAICT “each consideration is important enough to track that you should pay attention to them periodically.” And it feels like attempts to pin this down further feel more like some kind of culture war that isn’t primarily about the object-level fact of how often they are useful.
(apologies if I have missed a major point here, replying quickly at a busy time)