Eliezer wrote this in a private Facebook thread February 2017:
Reminder: Eliezer and Holden are both on record as saying that “steelmanning” people is bad and you should stop doing it.
As Holden says, if you’re trying to understand someone or you have any credence at all that they have a good argument, focus on passing their Ideological Turing Test. “Steelmanning” usually ends up as weakmanning by comparison. If they don’t in fact have a good argument, it’s falsehood to pretend they do. If you want to try to make a genuine effort to think up better arguments yourself because they might exist, don’t drag the other person into it.
Be it clear: Steelmanning is not a tool of understanding and communication. The communication tool is the Ideological Turing Test. “Steelmanning” is what you do to avoid the equivalent of dismissing AGI after reading a media argument. It usually indicates that you think you’re talking to somebody as hapless as the media.
The exception to this rule is when you communicate, “Well, on my assumptions, the plausible thing that sounds most like this is...” which is a cooperative way of communicating to the person what your own assumptions are and what you think are the strong and weak points of what you think might be the argument.
Mostly, you should be trying to pass the Ideological Turing Test if speaking to someone you respect, and offering “My steelman might be...?” only to communicate your own premises and assumptions. Or maybe, if you actually believe the steelman, say, “I disagree with your reason for thinking X, but I’ll grant you X because I believe this other argument Y. Is that good enough to move on?” Be ready to accept “No, the exact argument for X is important to my later conclusions” as an answer.
“Let me try to imagine a smarter version of this stupid position” is when you’ve been exposed to the Deepak Chopra version of quantum mechanics, and you don’t know if it’s the real version, or what a smart person might really think is the issue. It’s what you do when you don’t want to be that easily manipulated sucker who can be pushed into believing X by a flawed argument for not-X that you can congratulate yourself for being skeptically smarter than. It’s not what you do in a respectful conversation.
It seems like in the vast majority of conversations, we find ourselves closer to the “exposed to the Deepak Chopra version of quantum mechanics and haven’t seen the actual version yet” situation than we do to the “Arguing with someone who is far less experienced and knowledgeable than you are on this subject.” In the latter case, it’s easy to see why steelmanning would be counterproductive. If you’re a professor trying to communicate a difficult subject to a student, and the student is having trouble understanding your position, it’s unhelpful to try to “steelman” the student (i.e. try to present a logical-sounding but faulty argument in favor of what the student is saying), but it’s far more helpful to the student to try to “pass their ITT” by modeling their confusions and intuitions, and then use that to try to help them understand the correct argument. I can imagine Eliezer and Holden finding themselves in this situation more often than not, since they are both experts in their respective fields and have spent many years refining their reasoning skills and fine-tuning the arguments to their various positions on things.
But in most situations, for most of us who may not quite know how strong the epistemological ground we stand on really is, are probably using some mixture of flawed intuitions and logic to present our understandings of some topic. We might also be modeling people whom we really respect as being in a similar situation as we are. In which case it seems like the line between steelmanning and ITT becomes a bit blurry. If I know that both of us are using some combination of intuition (prone to bias and sometimes hard to describe), importance weighting of various facts, and different logical pathways to reach some set of conclusions, both trying to pass each other’s ITT as well as steelmanning potentially have some utility. The former might help to iron out differences in our intuitions and harder to formalize disagreements, and the latter might help with actually reaching more formal versions of arguments, or reasoning paths that have yet to be explored.
But I do find it easy to imagine that as I progress in my understanding and expertise in some particular topic, the benefits of steelmanning relative to ITT do seem to decrease. But it’s not clear to me that I (or anyone outside of the areas they spend most of their time thinking about) have actually reached this point in situations where we are debating with or cooperating on a problem together with respected peers.
Eliezer wrote this in a private Facebook thread February 2017:
And he FB-commented on Ozy’s Against Steelmanning in August 2016:
It seems like in the vast majority of conversations, we find ourselves closer to the “exposed to the Deepak Chopra version of quantum mechanics and haven’t seen the actual version yet” situation than we do to the “Arguing with someone who is far less experienced and knowledgeable than you are on this subject.” In the latter case, it’s easy to see why steelmanning would be counterproductive. If you’re a professor trying to communicate a difficult subject to a student, and the student is having trouble understanding your position, it’s unhelpful to try to “steelman” the student (i.e. try to present a logical-sounding but faulty argument in favor of what the student is saying), but it’s far more helpful to the student to try to “pass their ITT” by modeling their confusions and intuitions, and then use that to try to help them understand the correct argument. I can imagine Eliezer and Holden finding themselves in this situation more often than not, since they are both experts in their respective fields and have spent many years refining their reasoning skills and fine-tuning the arguments to their various positions on things.
But in most situations, for most of us who may not quite know how strong the epistemological ground we stand on really is, are probably using some mixture of flawed intuitions and logic to present our understandings of some topic. We might also be modeling people whom we really respect as being in a similar situation as we are. In which case it seems like the line between steelmanning and ITT becomes a bit blurry. If I know that both of us are using some combination of intuition (prone to bias and sometimes hard to describe), importance weighting of various facts, and different logical pathways to reach some set of conclusions, both trying to pass each other’s ITT as well as steelmanning potentially have some utility. The former might help to iron out differences in our intuitions and harder to formalize disagreements, and the latter might help with actually reaching more formal versions of arguments, or reasoning paths that have yet to be explored.
But I do find it easy to imagine that as I progress in my understanding and expertise in some particular topic, the benefits of steelmanning relative to ITT do seem to decrease. But it’s not clear to me that I (or anyone outside of the areas they spend most of their time thinking about) have actually reached this point in situations where we are debating with or cooperating on a problem together with respected peers.