FWIW it’s not TOTALLY obvious to me that the literature supports the notion that deliberate practice applies to meta-cognitive skills at the highest level like this.
Evidence for this type of universal transfer learning is scant.
It’s clear to me from my own experience that this can be done, but if people are like “ok buddy, you SAY you’ve used focused techniques and practice to be more productive, but I think you just grew out of your ADHD” (which people HAVE said to me), I don’t think it’s fair to just say “cummon man, deliberate practice works!”
I think your second objection is actually very strong
Okay, but I don’t think you can practice at “thinking”
Not because you’ll goodhart, but because people think it’s plausible that the mind just isn’t plastic on this level of basic meta-cognition. There’s lots of evidence AGAINST that, and many times that people THINK they’ve find this sort of universal transfer, it often ends up being more domain specific than they thought.
Probably the most compelling evidence that it’s possible is spiritual traditions (especially empirical ones like Buddhism) that consistently show that through a specific method, you can get deep shifts in ways of seeing and relating to everything that are consistently described the same way by many different people over space and time. But in terms of the experimental literature I don’t think there actually IS much good support for universal transfer of meta-cognition via deliberate practice.
Okay, yeah this should have been dealt with in the OP. I have thoughts about this but I did write the essay in a bit of a rush. I agree this is one of the strongest objections.
I had someone do some review of the transfer learning literature. There was nonzero stuff there seemed to demonstrably work. But mostly it just seemed like we just don’t really have good experiments on the stuff I’d have expected to work. (And, the sorts of experiments that I’d expect to work are quite expensive)
But I don’t think “universal transfer learning” is quite the phrase here.
If you learn arithmetic, you (probably) don’t get better at arbitrary other skills. But, you get to use arithmetic wherever it’s relevant. You do have to separately practice “notice places where arithmetic is relevant.” (like it may not occur to you that many problems you face are actually math problem. Or, you might need an additional skill like Fermi Estimation to turn them into math problems)
The claim here is more like: “Noticing confusion”, “having more than 1 hypothesis”, “noticing yourself flinching away from a thought that’d be inconvenient” are skills that show up in multiple domains.
My “c’mon guys” here is not “c’mon the empirical evidence here is overwhelming.” It’s more like “look, which world do you actually expect to result in you making better decisions faster: the one where you spend >0 days on testing and reflecting on your thinking in areas where there is real feedback, or the one where you just spend all your time on ‘object level work’ that doesn’t really have the ability to tell you you were wrong?”.
(and, a host of similar questions, with the meta question is “do you really expect the optimal thing here to be zero effort on metacognition practice of some kind?”)
Obviously there is a question of how much time to spend on this is optimal, and it’s definitely possible (and perhaps common) to go overboard. But I also think it’s not too hard to figure out how to navigate that.
My “c’mon guys” here is not “c’mon the empirical evidence here is overwhelming.” It’s more like “look, which world do you actually expect to result in you making better decisions faster: the one where you spend >0 days on testing and reflecting on your thinking in areas where there is real feedback, or the one where you just spend all your time on ‘object level work’ that doesn’t really have the ability to tell you you were wrong?”.
(and, a host of similar questions, with the meta question is “do you really expect the optimal thing here to be zero effort on metacognition practice of some kind?”)
I mostly agree in general and I feel ya on the “c’mon guys” thing, yet I don’t do my own separate “rationality practice”.
For me, it’s basically the same reason why I don’t spend much time in a weight room anymore; I prefer to keep my strength by doing things that require and use strength. I’m not against weight lifting in principle, and I’ve done a decent amount of it. It’s just that when I have a choice between “exercise muscles for the sake of exercising muscles” and “exercise muscles in the process of doing something else I want to do anyway”, the latter is a pure win if the exercise is anywhere near equivalent. Not only is it “two birds with one stone”, it also streamlines the process of making sure you’re training the right muscles for the uses you actually have, and streamlines the process of maintaining motivation with proof that it is concretely useful.
The option isn’t always available, obviously. If your object level work doesn’t have good feedback, or you’re not strong enough to do your job, then specific training absolutely makes sense. Personally though, I find more than enough opportunities to work on meta cognition as applied to actual things I am doing for object level reasons.
The thing that seems more important to me isn’t whether you’re doing a separate practice for the sake of learning, but whether you’re reflecting on your thinking in areas where there’s real feedback, and you’re noticing that feedback. I do think there’s a place for working on artificial problems, but I also think there’s an under recognized place for picking the right real world problems for your current ability level with an expectation of learning to level up. And an underappreciated skill in finding feedback on less legible problems.
FWIW it’s not TOTALLY obvious to me that the literature supports the notion that deliberate practice applies to meta-cognitive skills at the highest level like this.
Evidence for this type of universal transfer learning is scant.
It’s clear to me from my own experience that this can be done, but if people are like “ok buddy, you SAY you’ve used focused techniques and practice to be more productive, but I think you just grew out of your ADHD” (which people HAVE said to me), I don’t think it’s fair to just say “cummon man, deliberate practice works!”
I think your second objection is actually very strong
Not because you’ll goodhart, but because people think it’s plausible that the mind just isn’t plastic on this level of basic meta-cognition. There’s lots of evidence AGAINST that, and many times that people THINK they’ve find this sort of universal transfer, it often ends up being more domain specific than they thought.
Probably the most compelling evidence that it’s possible is spiritual traditions (especially empirical ones like Buddhism) that consistently show that through a specific method, you can get deep shifts in ways of seeing and relating to everything that are consistently described the same way by many different people over space and time. But in terms of the experimental literature I don’t think there actually IS much good support for universal transfer of meta-cognition via deliberate practice.
Okay, yeah this should have been dealt with in the OP. I have thoughts about this but I did write the essay in a bit of a rush. I agree this is one of the strongest objections.
I had someone do some review of the transfer learning literature. There was nonzero stuff there seemed to demonstrably work. But mostly it just seemed like we just don’t really have good experiments on the stuff I’d have expected to work. (And, the sorts of experiments that I’d expect to work are quite expensive)
But I don’t think “universal transfer learning” is quite the phrase here.
If you learn arithmetic, you (probably) don’t get better at arbitrary other skills. But, you get to use arithmetic wherever it’s relevant. You do have to separately practice “notice places where arithmetic is relevant.” (like it may not occur to you that many problems you face are actually math problem. Or, you might need an additional skill like Fermi Estimation to turn them into math problems)
The claim here is more like: “Noticing confusion”, “having more than 1 hypothesis”, “noticing yourself flinching away from a thought that’d be inconvenient” are skills that show up in multiple domains.
My “c’mon guys” here is not “c’mon the empirical evidence here is overwhelming.” It’s more like “look, which world do you actually expect to result in you making better decisions faster: the one where you spend >0 days on testing and reflecting on your thinking in areas where there is real feedback, or the one where you just spend all your time on ‘object level work’ that doesn’t really have the ability to tell you you were wrong?”.
(and, a host of similar questions, with the meta question is “do you really expect the optimal thing here to be zero effort on metacognition practice of some kind?”)
Obviously there is a question of how much time to spend on this is optimal, and it’s definitely possible (and perhaps common) to go overboard. But I also think it’s not too hard to figure out how to navigate that.
I mostly agree in general and I feel ya on the “c’mon guys” thing, yet I don’t do my own separate “rationality practice”.
For me, it’s basically the same reason why I don’t spend much time in a weight room anymore; I prefer to keep my strength by doing things that require and use strength. I’m not against weight lifting in principle, and I’ve done a decent amount of it. It’s just that when I have a choice between “exercise muscles for the sake of exercising muscles” and “exercise muscles in the process of doing something else I want to do anyway”, the latter is a pure win if the exercise is anywhere near equivalent. Not only is it “two birds with one stone”, it also streamlines the process of making sure you’re training the right muscles for the uses you actually have, and streamlines the process of maintaining motivation with proof that it is concretely useful.
The option isn’t always available, obviously. If your object level work doesn’t have good feedback, or you’re not strong enough to do your job, then specific training absolutely makes sense. Personally though, I find more than enough opportunities to work on meta cognition as applied to actual things I am doing for object level reasons.
The thing that seems more important to me isn’t whether you’re doing a separate practice for the sake of learning, but whether you’re reflecting on your thinking in areas where there’s real feedback, and you’re noticing that feedback. I do think there’s a place for working on artificial problems, but I also think there’s an under recognized place for picking the right real world problems for your current ability level with an expectation of learning to level up. And an underappreciated skill in finding feedback on less legible problems.