I followed up on this with a year exploring various rationality exercises and workshops. My plans and details have evolved a bunch since then, but I still think the opening 7 bullets (i.e. “Deliberate Practice, Metacognition” etc, with “actually do the goddamn practice” and “the feedbackloop is the primary product”) are quite important guiding lights.
I’ve written up most of my updates as they happened over the year, in:
The primary aim of my workshops and practice is to get new skills fluent enough that you can appy them to your day job, because that’s where it’s practical to deliberate practice in a way that pays for itself rather than being an exhausting extra thing you do.
“Fluency at new skill that seem demonstrably useful” is also a large enough effect size that there’s at least something you can measure near term, to get a sense of whether the workshop is working.
Five minute versions of skills.
Relatedly: many skills have elaborate, comprehensive versions that take ~an hour to get the full value of, but you’re realisitically not going to do those most of the time. So it’s important to boil them down into something you can do in 5 minutes (or, 30 seconds).
Morning Orient Prompts.
A thing I’ve find useful for myself, and now think of as one of the primary goal of the workshop to get people to try out, is a “morning orient prompt list” that you do every day.
It’s important that it be every day, even when you don’t need it too much, so that you still have a habit of metacognition for the times you need it (but, when you don’t need it too much, it’s fine/good to do a very quick version of it)
It’s useful to have a list of explicit prompts, because that gives you an artifact that’s easier to iterate on.
What about RCT-style science?
The original post noted an expensive plan that would be “actual science shaped”, which would give a clearer sense of whether rationality training worked. I do still think that’s the right goal to be aspiring to.
It is quite an expensive goal – not just in my effort but in a lot of smart people’s time.
Oddly, the two major objections I frequently get are both “why haven’t you done an RCT?” and “Doing [the real RCT version of this] is too expensive so the plan is bad.”
My actual belief right now is that the most important feedbackloops to be developing are more focused on “how to track if this is helping with your day-job”, because I think that’s what most people’s cruxes actually are (and, should be). RCTs would at best tell you what works on average for People of Some Reference Class, but still don’t actually tell you personally if it’s worthwhile for you.
A lot of my focus right now is helping people who have attended a workshop to integrate the practice into their daily life, and seeing what sort of problems come up there and how to deal with them. (Where my current partial-answers to that are the previous three sections)
I followed up on this with a year exploring various rationality exercises and workshops. My plans and details have evolved a bunch since then, but I still think the opening 7 bullets (i.e. “Deliberate Practice, Metacognition” etc, with “actually do the goddamn practice” and “the feedbackloop is the primary product”) are quite important guiding lights.
I’ve written up most of my updates as they happened over the year, in:
Rationality Research Report: Towards 10x OODA Looping?
“Fractal Strategy” workshop report
The Cognitive Bootcamp Agreement
Skills from a year of Purposeful Rationality Practice
The biggest overall updates since this post:
Fluent enough for your day job.
The primary aim of my workshops and practice is to get new skills fluent enough that you can appy them to your day job, because that’s where it’s practical to deliberate practice in a way that pays for itself rather than being an exhausting extra thing you do.
“Fluency at new skill that seem demonstrably useful” is also a large enough effect size that there’s at least something you can measure near term, to get a sense of whether the workshop is working.
Five minute versions of skills.
Relatedly: many skills have elaborate, comprehensive versions that take ~an hour to get the full value of, but you’re realisitically not going to do those most of the time. So it’s important to boil them down into something you can do in 5 minutes (or, 30 seconds).
Morning Orient Prompts.
A thing I’ve find useful for myself, and now think of as one of the primary goal of the workshop to get people to try out, is a “morning orient prompt list” that you do every day.
It’s important that it be every day, even when you don’t need it too much, so that you still have a habit of metacognition for the times you need it (but, when you don’t need it too much, it’s fine/good to do a very quick version of it)
It’s useful to have a list of explicit prompts, because that gives you an artifact that’s easier to iterate on.
What about RCT-style science?
The original post noted an expensive plan that would be “actual science shaped”, which would give a clearer sense of whether rationality training worked. I do still think that’s the right goal to be aspiring to.
It is quite an expensive goal – not just in my effort but in a lot of smart people’s time.
Oddly, the two major objections I frequently get are both “why haven’t you done an RCT?” and “Doing [the real RCT version of this] is too expensive so the plan is bad.”
My actual belief right now is that the most important feedbackloops to be developing are more focused on “how to track if this is helping with your day-job”, because I think that’s what most people’s cruxes actually are (and, should be). RCTs would at best tell you what works on average for People of Some Reference Class, but still don’t actually tell you personally if it’s worthwhile for you.
A lot of my focus right now is helping people who have attended a workshop to integrate the practice into their daily life, and seeing what sort of problems come up there and how to deal with them. (Where my current partial-answers to that are the previous three sections)