Valid question. I think the way I framed it may over-sell the solution—there’ll still be some problems of reference-class choice. But, you should move toward gears to try and resolve that.
The way in which policy-level thinking improves the reference-class tennis is that you’re not stopping at reference-class reasoning. You’re trying to map the input-output relations involved.
If you think you’re terrible at planning, but not in cases X, Y, and Z, what mechanisms do you think lie behind that? You don’t have to know in order to use the reference classes, but if you are feeling uncertain about the validity of the reference classes, digging down into causal details is likely a pretty good way to disambiguate.
For example, maybe you find that you’re tempted to say something very critical of someone (accuse them of lying), but you notice that you’re in a position where you are socially incentivised to do so (everyone is criticising this person and you feel like joining in). However, you alse value honesty, and don’t want to accuse them of lying unfairly. You don’t think you’re being dishonest with yourself about it, but you can remember other situations where you’ve joined in an group criticism and later realized you were unfair due to the social momentum.
I think the reference-class reaction is to just downgrade your certainty, and maybe have a policy of not speaking up in those kinds of situations. This isn’t a bad reaction, but it can be seen as a sort of epistemic learned helplessness. “I’ve been irrational in this sort of situation, therefore I’m incapable of being rational in this sort of situation.” You might end up generally uncomfortable with this sort of social situation and feeling like you don’t know how to handle it well.
So, another reaction, which might be better in the long-term, would be to take a look at your thinking process. “What makes me think they’re a liar? Wow, I’m not working on much evidence here. There are all kinds of alternative explanations, I just gravitated to that one...”
It’s a somewhat subtle distinction there; maybe not the clearest example.
Another example which is pretty clear: someone who you don’t like provides a critique of your idea. You are tempted to reject it out of hand, but outside view tells you you’re likely to reject that person’s comments reguardless of truth. A naive adjustment might be to upgrade your credence in their criticism. This seems like something you only want to do if your reasoning in the domain is too messy to assess objectively. Policy-level thinking might say that the solution is to judge their critique on its merits. Your “inside view” (in the sense of how-reality-naively-seems-to-you) is that it’s a bad critique; but this obviously isn’t a gears-level assessment.
Maybe your policy-level analysis is that you’ll be unable to judge a critique objectively in such cases, even if you pause to try and imagine what you’d think of it if you came up with it yourself. In that case, maybe you decide that what you’ll do in such cases is write it down and think it through in more detail later (and say as much to the critic).
Or, maybe your best option really is to downgrade your own belief without assessing the critique at the gears level. (Perhaps this issue isn’t that important to you, and you don’t want to spend time evaluating arguments in detail.) But the point is that you can go into more detail.
Doesn’t this basically move the reference class tennis to the meta-level?
“Oh, in general I’m terrible at planning, but not in cases involving X, Y and Z!”
It seems reasonable that this is harder to do this on a meta-level, but do any of the other points you mention actually “solve” this problem?
Valid question. I think the way I framed it may over-sell the solution—there’ll still be some problems of reference-class choice. But, you should move toward gears to try and resolve that.
The way in which policy-level thinking improves the reference-class tennis is that you’re not stopping at reference-class reasoning. You’re trying to map the input-output relations involved.
If you think you’re terrible at planning, but not in cases X, Y, and Z, what mechanisms do you think lie behind that? You don’t have to know in order to use the reference classes, but if you are feeling uncertain about the validity of the reference classes, digging down into causal details is likely a pretty good way to disambiguate.
For example, maybe you find that you’re tempted to say something very critical of someone (accuse them of lying), but you notice that you’re in a position where you are socially incentivised to do so (everyone is criticising this person and you feel like joining in). However, you alse value honesty, and don’t want to accuse them of lying unfairly. You don’t think you’re being dishonest with yourself about it, but you can remember other situations where you’ve joined in an group criticism and later realized you were unfair due to the social momentum.
I think the reference-class reaction is to just downgrade your certainty, and maybe have a policy of not speaking up in those kinds of situations. This isn’t a bad reaction, but it can be seen as a sort of epistemic learned helplessness. “I’ve been irrational in this sort of situation, therefore I’m incapable of being rational in this sort of situation.” You might end up generally uncomfortable with this sort of social situation and feeling like you don’t know how to handle it well.
So, another reaction, which might be better in the long-term, would be to take a look at your thinking process. “What makes me think they’re a liar? Wow, I’m not working on much evidence here. There are all kinds of alternative explanations, I just gravitated to that one...”
It’s a somewhat subtle distinction there; maybe not the clearest example.
Another example which is pretty clear: someone who you don’t like provides a critique of your idea. You are tempted to reject it out of hand, but outside view tells you you’re likely to reject that person’s comments reguardless of truth. A naive adjustment might be to upgrade your credence in their criticism. This seems like something you only want to do if your reasoning in the domain is too messy to assess objectively. Policy-level thinking might say that the solution is to judge their critique on its merits. Your “inside view” (in the sense of how-reality-naively-seems-to-you) is that it’s a bad critique; but this obviously isn’t a gears-level assessment.
Maybe your policy-level analysis is that you’ll be unable to judge a critique objectively in such cases, even if you pause to try and imagine what you’d think of it if you came up with it yourself. In that case, maybe you decide that what you’ll do in such cases is write it down and think it through in more detail later (and say as much to the critic).
Or, maybe your best option really is to downgrade your own belief without assessing the critique at the gears level. (Perhaps this issue isn’t that important to you, and you don’t want to spend time evaluating arguments in detail.) But the point is that you can go into more detail.