Ah, yes, that is indeed the first thing one should work on, otherwise the MW (Must Win) interpretation of Rationality is little better than the MW (Many Worlds) interpretation of Quantum Mechanics. I didn’t realize that, after all this time, there are still no objective metrics to measure the success of the course. I wish I had good ideas as to how to experimentally measure rationality, but alas. Hopefully other forum regulars do. Or maybe EY can spend some time thinking about it.
I guess an obvious way to start is to score a particular behavior based on some objective criteria, like the pass/fail on those sunk cost situations Anna (?) linked here some time ago. It’s not nearly as good as actually putting people into the circumstances where they have to apply their newly learned skills (such as detecting confusion, recognizing cognitive dissonance, what have you), but it’s a start.
As a next step, my guess is that if you look through the standard psychological experiments (maybe something less drastic and notorious than the Stanford prison experiment), you will find quite a number of them that can be cheaply replicated in a controlled setting like a mini-camp. I’m sure that gwern can dig up a whole whack of them in no time flat. Or maybe you are already doing this, for all I know. The important thing is that the participants should be inside the situations, not outside of them, and hopefully unaware that they are being tested. I guess it is sort of similar to giving two sets of CRTs, before and after.
Ah, yes, that is indeed the first thing one should work on, otherwise the MW (Must Win) interpretation of Rationality is little better than the MW (Many Worlds) interpretation of Quantum Mechanics. I didn’t realize that, after all this time, there are still no objective metrics to measure the success of the course. I wish I had good ideas as to how to experimentally measure rationality, but alas. Hopefully other forum regulars do. Or maybe EY can spend some time thinking about it.
I guess an obvious way to start is to score a particular behavior based on some objective criteria, like the pass/fail on those sunk cost situations Anna (?) linked here some time ago. It’s not nearly as good as actually putting people into the circumstances where they have to apply their newly learned skills (such as detecting confusion, recognizing cognitive dissonance, what have you), but it’s a start.
As a next step, my guess is that if you look through the standard psychological experiments (maybe something less drastic and notorious than the Stanford prison experiment), you will find quite a number of them that can be cheaply replicated in a controlled setting like a mini-camp. I’m sure that gwern can dig up a whole whack of them in no time flat. Or maybe you are already doing this, for all I know. The important thing is that the participants should be inside the situations, not outside of them, and hopefully unaware that they are being tested. I guess it is sort of similar to giving two sets of CRTs, before and after.