I agree with your distaste given my understanding of ‘steelmanning’, which is something like “take a belief or position and imagine really good arguments for that” or “take an argument and make a different, better argument out of it” (i.e. the opposite of strawmanning), primarily because it takes you further away from what the person is saying (or at least poses an unacceptably high risk of that). That being said, the concrete suggestions under the heading of steelmanning, addressing core points and putting in interpretive effort, seem crucially different in that they bring you closer to what somebody is saying. As such, and unlike steelmanning, they seem to me like important parts of how one ought to engage in intellectual discussion.
Yeah, thinking overnight and reading more comments this morning has me updating that I shouldn’t have used the word steelmanning there, and I’ll update it soon (although I’m not 100% sure what the best term here is and not sure there’s a single term that does what I want).
Background:
When discussing this with other mods, I suggested saying that LW should be about collaborative rather than adversarial truthseeking, and other mods noted that there is a time for adversarial truthseeking even on LW and it’d probably be epistemically fraught to try and bake collaboration into the LW discussion DNA.
I ended up writing the three bullet points (address core points, invest interpretive labor, and being kind) as standalone points. It seemed important to distinguish the first two points from the last one. I searched my brain for a term that seemed to encompass the first two terms, generated “Steelman” and then called it a day. But, yeah, steelman has other properties that don’t quite make sense for what I’m trying to point at here.
I feel like there was a piece here that charity doesn’t quite address, where I actually _did_ mean something closer (but perhaps not identical to) steelman.
Elsethread, Vlad notes that steelman often ends up replacing someone’s argument with a totally different argument. This part is bad for purposes of communication, since you might end up misunderstanding someone’s position. But I think it’s good for purposes of goal-directed-discussion.
i.e. in my mind, the point of the discussion is to output something useful. If the something useful is different than what the author originally intended, that’s fine. (This is where “collaborative discussion” felt more right to me than most other terms)
Yes, I think the word ‘steelmanning’ is often used to cover some nice similar-ish conversational norms, and find it regrettable that I don’t know a better word off the top of my head. Perhaps it’s time to invent one?
Your criticism of steelmanning is apt. Of “interpretive effort” I have already spoken elsewhere. As for “addressing core points”, however…
The problem with this criterion is simply that an author and a reader may disagree on what the core points are.
Note that I am not simply talking about cases where a reader misunderstands what the author wrote, and mistakes something for an intended “core point” that is no such thing! Such cases are, in a sense, trivial, insofar as clearing up the misunderstanding results in the original piece standing unmodified (excepting, perhaps, any clarifying modifications that are aimed at preventing exactly such misunderstanding from reoccurring in other readers).
Rather, I’m talking about cases where the reader understands what the author is saying, but considers the author’s claimed “core point” to actually be of peripheral significance, and considers a different point (one which the author either did not mention at all, or devoted only scant attention to) to be central. In this case, no misunderstanding, per se, is occurring (unless you call a deep conceptual or empirical error a “misunderstanding”, which I do not—I reserve the latter term for miscommunication).
This sort of thing can come in many forms. I will describe just one of them here.
It sometimes happens that someone—call him Dave—will write a post describing some (purported) phenomenon P—some pattern, some dynamic, some causal structure, etc.—and, to illustrate P, will provide an example E.
Carol, a reader, then comes along and says: “Actually, Dave, I don’t think E is an example of P! Consider the following…” Dave perhaps defends his example, perhaps not, but agrees, in the end, that E is not really a good example of P after all. “But,” says Dave, “that was just an example! Quit nitpicking, Carol—address my core points!”
What is Carol to say to this? Does Dave accuse her fairly? Is it mere nitpicking to attack a mere example?
But if E is the only example of P that Dave had provided, then—E having now been disqualified from that role—Dave is left with no examples of P. And if Dave cannot provide any other (or, I should simply say, any) examples of P, then perhaps P is simply not real? I can hardly thinking of a more “core” point than the question of whether the thing we’re talking about is even a real thing!
As I say, the “attacking an example” form is just one of the ways in which the “disagreement about what the core points are” dynamic can manifest. But it alone has, in my view, been responsible for a great deal of epistemic trouble among rationalists. As I’ve said elsewhere, we—the types of people who frequent such sites as Less Wrong—are very good at inventing abstract patterns, “crystallizing concepts”, constructing systems of classification, and so on. We are so good at it, in fact, that this talent of ours often gets away from us. It is absolutely critical to keep it reined in. (I do not choose the expression lightly! Reins, after all, are used, not to prevent a horse from moving, but to make it move in the direction you want to go—instead of uselessly running wild.) And those reins are made of real-world examples, they are made of extensions (as vs. intensions), they are made of practice. Without it, deceiving ourselves that we’ve gained knowledge, when in fact we’re building sky castles of abstract nonsense, is all too easy.
Perhaps you have other examples of dynamics where what the ‘core points’ are is in dispute, but the Carol and Dave case seems like one where there’s just a sort of miscommunication: Dave thinks ‘whether E is an example of P’ is not a core point, Carol thinks (I presume) ‘whether there exist any examples of P’ is a core point, and it seems likely to me that both of these can be true and agreed upon by both parties. I’d imagine that if Carol’s initial comment were ‘I don’t think E is an example of P, because … Also, I doubt that there are any examples of P at all—could you give another one, or address my misgivings about E?’ or instead ‘Despite thinking that there are many examples of P, I don’t think E is one, because …’ then there wouldn’t be a dispute about whether core points were being addressed.
Dave thinks ‘whether E is an example of P’ is not a core point, Carol thinks (I presume) ‘whether there exist any examples of P’ is a core point, and it seems likely to me that both of these can be true and agreed upon by both parties.
These can both be true. But they also may not both be true—for instance, in cases where E is representative of a class of pseudo-examples (i.e., scenarios that have the property of seeming to be examples of P but not actually being examples of P). Similarly, ‘whether E is an example of P’ is often indeed a core point in virtue of ‘if E is not an example of P, why did Dave think that it is?’ being a core point; the latter question often goes to the heart of Dave’s view, and his reasons for holding it!
It also happens to empirically be the case that many (perhaps, most?) real-life analogues of Dave do not consider ‘whether there exist any examples of P’ to be a core point of their claims (or, at least, that is the strong impression one gets from the way in which they respond to inquiries about examples).
Finally, ‘does Dave have any actual examples of P’ is a very strong indicator—strong Bayesian evidence, if you want to view it that way—of whether we ought to believe P, or how seriously we ought to take Dave’s claims. (No, “just evaluate Dave’s argument, aside from examples” is not an acceptable response to this!)
… if Carol’s initial comment were ‘I don’t think E is an example of P, because … Also, I doubt that there are any examples of P at all—could you give another one, or address my misgivings about E?’ …
Doubting that there are any examples of P is not, so to speak, Carol’s job. The claim is that E is an example of P. The only reason Carol has for thinking that there are examples of P (excepting cases where P is something well-known, of which there are obviously many examples) is that Dave has described E to the reader. Once E is disqualified, Carol is back to having no particular reason to believe that there are any examples of P.
Once E is disqualified, it is (or it ought to be!) implied that supplying other examples of P is now incumbent upon Dave. Carol bears no obligation (either epistemic or rhetorical) to commit to any position on the question of “are there any examples of P”, in order for Dave to be faced with the need to provide replacement examples.
In short, I think that “having made a claim, has Dave in fact provided any actual examples of the claimed thing” is (barring edge cases) always a core point.
Doubting that there are any examples of P is not, so to speak, Carol’s job. The claim is that E is an example of P. The only reason Carol has for thinking that there are examples of P (excepting cases where P is something well-known, of which there are obviously many examples) is that Dave has described E to the reader. Once E is disqualified, Carol is back to having no particular reason to believe that there are any examples of P.
It seems to me that there’s likely to be enough cases where there are differences in opinion about whether P is well-known enough that examples aren’t needed, or whether P isn’t well-known but whether the reader upon hearing a definition could think of examples themselves, that it’s useful to have norms whereby we clarify whether or not we doubt that there examples of P.
All of the cases I am thinking of are those where P is a new concept, which the author is defining / describing / “crystallizing” for the first time. As such, it seems unlikely that this sort of edge case would apply.
I do agree that working examples are quite important (and that this is something authors should be encouraged to provide).
The issue I expect to be relevant to your past experience is different takes on which examples are valid.
(My impression is that it is often the case, esp. with discussions relating to internal mental states, that the author provides something that makes total sense to them, and me, but doesn’t make sense to you, and then you continuously ask for better/different examples when it seems like the underlying issue is that for whatever reason, the particular mental phenomena their referencing isn’t relevant to you.
How to resolve this seems like a different question that the rest of this thread is focusing on. But I think by this point a lot of people not-providing-you-in-particular-with-examples is because they don’t expect your criticism of their examples to be that useful.)
Oh, certainly this is a fair point. No argument there! But we can agree, I think, that “you say E is not an example of P, but I maintain that it is” is not at all the same thing as “you’re not addressing the core point”—yes?
I agree with your distaste given my understanding of ‘steelmanning’, which is something like “take a belief or position and imagine really good arguments for that” or “take an argument and make a different, better argument out of it” (i.e. the opposite of strawmanning), primarily because it takes you further away from what the person is saying (or at least poses an unacceptably high risk of that). That being said, the concrete suggestions under the heading of steelmanning, addressing core points and putting in interpretive effort, seem crucially different in that they bring you closer to what somebody is saying. As such, and unlike steelmanning, they seem to me like important parts of how one ought to engage in intellectual discussion.
Yeah, thinking overnight and reading more comments this morning has me updating that I shouldn’t have used the word steelmanning there, and I’ll update it soon (although I’m not 100% sure what the best term here is and not sure there’s a single term that does what I want).
Background:
When discussing this with other mods, I suggested saying that LW should be about collaborative rather than adversarial truthseeking, and other mods noted that there is a time for adversarial truthseeking even on LW and it’d probably be epistemically fraught to try and bake collaboration into the LW discussion DNA.
I ended up writing the three bullet points (address core points, invest interpretive labor, and being kind) as standalone points. It seemed important to distinguish the first two points from the last one. I searched my brain for a term that seemed to encompass the first two terms, generated “Steelman” and then called it a day. But, yeah, steelman has other properties that don’t quite make sense for what I’m trying to point at here.
It sounds like principle of charity is a better match for your intended meaning than steelman.
(not an official mod take, me thinking out loud)
I feel like there was a piece here that charity doesn’t quite address, where I actually _did_ mean something closer (but perhaps not identical to) steelman.
Elsethread, Vlad notes that steelman often ends up replacing someone’s argument with a totally different argument. This part is bad for purposes of communication, since you might end up misunderstanding someone’s position. But I think it’s good for purposes of goal-directed-discussion.
i.e. in my mind, the point of the discussion is to output something useful. If the something useful is different than what the author originally intended, that’s fine. (This is where “collaborative discussion” felt more right to me than most other terms)
Yes, I think the word ‘steelmanning’ is often used to cover some nice similar-ish conversational norms, and find it regrettable that I don’t know a better word off the top of my head. Perhaps it’s time to invent one?
Your criticism of steelmanning is apt. Of “interpretive effort” I have already spoken elsewhere. As for “addressing core points”, however…
The problem with this criterion is simply that an author and a reader may disagree on what the core points are.
Note that I am not simply talking about cases where a reader misunderstands what the author wrote, and mistakes something for an intended “core point” that is no such thing! Such cases are, in a sense, trivial, insofar as clearing up the misunderstanding results in the original piece standing unmodified (excepting, perhaps, any clarifying modifications that are aimed at preventing exactly such misunderstanding from reoccurring in other readers).
Rather, I’m talking about cases where the reader understands what the author is saying, but considers the author’s claimed “core point” to actually be of peripheral significance, and considers a different point (one which the author either did not mention at all, or devoted only scant attention to) to be central. In this case, no misunderstanding, per se, is occurring (unless you call a deep conceptual or empirical error a “misunderstanding”, which I do not—I reserve the latter term for miscommunication).
This sort of thing can come in many forms. I will describe just one of them here.
It sometimes happens that someone—call him Dave—will write a post describing some (purported) phenomenon P—some pattern, some dynamic, some causal structure, etc.—and, to illustrate P, will provide an example E.
Carol, a reader, then comes along and says: “Actually, Dave, I don’t think E is an example of P! Consider the following…” Dave perhaps defends his example, perhaps not, but agrees, in the end, that E is not really a good example of P after all. “But,” says Dave, “that was just an example! Quit nitpicking, Carol—address my core points!”
What is Carol to say to this? Does Dave accuse her fairly? Is it mere nitpicking to attack a mere example?
But if E is the only example of P that Dave had provided, then—E having now been disqualified from that role—Dave is left with no examples of P. And if Dave cannot provide any other (or, I should simply say, any) examples of P, then perhaps P is simply not real? I can hardly thinking of a more “core” point than the question of whether the thing we’re talking about is even a real thing!
As I say, the “attacking an example” form is just one of the ways in which the “disagreement about what the core points are” dynamic can manifest. But it alone has, in my view, been responsible for a great deal of epistemic trouble among rationalists. As I’ve said elsewhere, we—the types of people who frequent such sites as Less Wrong—are very good at inventing abstract patterns, “crystallizing concepts”, constructing systems of classification, and so on. We are so good at it, in fact, that this talent of ours often gets away from us. It is absolutely critical to keep it reined in. (I do not choose the expression lightly! Reins, after all, are used, not to prevent a horse from moving, but to make it move in the direction you want to go—instead of uselessly running wild.) And those reins are made of real-world examples, they are made of extensions (as vs. intensions), they are made of practice. Without it, deceiving ourselves that we’ve gained knowledge, when in fact we’re building sky castles of abstract nonsense, is all too easy.
Perhaps you have other examples of dynamics where what the ‘core points’ are is in dispute, but the Carol and Dave case seems like one where there’s just a sort of miscommunication: Dave thinks ‘whether E is an example of P’ is not a core point, Carol thinks (I presume) ‘whether there exist any examples of P’ is a core point, and it seems likely to me that both of these can be true and agreed upon by both parties. I’d imagine that if Carol’s initial comment were ‘I don’t think E is an example of P, because … Also, I doubt that there are any examples of P at all—could you give another one, or address my misgivings about E?’ or instead ‘Despite thinking that there are many examples of P, I don’t think E is one, because …’ then there wouldn’t be a dispute about whether core points were being addressed.
These can both be true. But they also may not both be true—for instance, in cases where E is representative of a class of pseudo-examples (i.e., scenarios that have the property of seeming to be examples of P but not actually being examples of P). Similarly, ‘whether E is an example of P’ is often indeed a core point in virtue of ‘if E is not an example of P, why did Dave think that it is?’ being a core point; the latter question often goes to the heart of Dave’s view, and his reasons for holding it!
It also happens to empirically be the case that many (perhaps, most?) real-life analogues of Dave do not consider ‘whether there exist any examples of P’ to be a core point of their claims (or, at least, that is the strong impression one gets from the way in which they respond to inquiries about examples).
Finally, ‘does Dave have any actual examples of P’ is a very strong indicator—strong Bayesian evidence, if you want to view it that way—of whether we ought to believe P, or how seriously we ought to take Dave’s claims. (No, “just evaluate Dave’s argument, aside from examples” is not an acceptable response to this!)
Doubting that there are any examples of P is not, so to speak, Carol’s job. The claim is that E is an example of P. The only reason Carol has for thinking that there are examples of P (excepting cases where P is something well-known, of which there are obviously many examples) is that Dave has described E to the reader. Once E is disqualified, Carol is back to having no particular reason to believe that there are any examples of P.
Once E is disqualified, it is (or it ought to be!) implied that supplying other examples of P is now incumbent upon Dave. Carol bears no obligation (either epistemic or rhetorical) to commit to any position on the question of “are there any examples of P”, in order for Dave to be faced with the need to provide replacement examples.
In short, I think that “having made a claim, has Dave in fact provided any actual examples of the claimed thing” is (barring edge cases) always a core point.
It seems to me that there’s likely to be enough cases where there are differences in opinion about whether P is well-known enough that examples aren’t needed, or whether P isn’t well-known but whether the reader upon hearing a definition could think of examples themselves, that it’s useful to have norms whereby we clarify whether or not we doubt that there examples of P.
All of the cases I am thinking of are those where P is a new concept, which the author is defining / describing / “crystallizing” for the first time. As such, it seems unlikely that this sort of edge case would apply.
I do agree that working examples are quite important (and that this is something authors should be encouraged to provide).
The issue I expect to be relevant to your past experience is different takes on which examples are valid.
(My impression is that it is often the case, esp. with discussions relating to internal mental states, that the author provides something that makes total sense to them, and me, but doesn’t make sense to you, and then you continuously ask for better/different examples when it seems like the underlying issue is that for whatever reason, the particular mental phenomena their referencing isn’t relevant to you.
How to resolve this seems like a different question that the rest of this thread is focusing on. But I think by this point a lot of people not-providing-you-in-particular-with-examples is because they don’t expect your criticism of their examples to be that useful.)
Oh, certainly this is a fair point. No argument there! But we can agree, I think, that “you say E is not an example of P, but I maintain that it is” is not at all the same thing as “you’re not addressing the core point”—yes?