I don’t know if it can be reliably measured—my impression is that it’s pretty damn hard, and that once you heard about most tests (the kind that are used to show irrationality in the first place, i.e. those already discussed on less wrong), you won’t fall for them but that doesn’t say much about how less likely you are for thinking right in “real life” situations where your brain isn’t primed by “this is a test of rationality”.
Some sub-components can be reliably measured—such as calibration. Any others?
My thought on hearing the proposal was that it would be impractically difficult, but on further consideration I suspect it would be much easier than creating a reliable test for intelligence. With proper effort, we should at least be able to beat the standard set by IQ tests
I got through unscathed, but I suspect most atheists would. I don’t think that test is very good at discerning rationality beyond the “knows about logic” and “doesn’t believe in God”.
FWIW, I started it with “God Exists” as true and also got through it unscathed. But you’re right, it doesn’t seem to try very hard presenting atheists with pitfalls and traps. I was at least expecting some kind of foul trickery about the Big Bang.
I would expect it to be extremely hard. I do not limit the scope of these tests to simple paper question and answer. I would be ok if an MRI was needed, for the teste not to know the test was about rationality, if it required a group of well trained actors to fool the teste, or a virtual simulation(within todays technology). Of course tests that can be used/verified immediately by users are preferable.
In the end the techniques required may be prohibitive, but at least it would make known what techniques/technologies to watch/promote for the future.
I don’t know if it can be reliably measured—my impression is that it’s pretty damn hard, and that once you heard about most tests (the kind that are used to show irrationality in the first place, i.e. those already discussed on less wrong), you won’t fall for them but that doesn’t say much about how less likely you are for thinking right in “real life” situations where your brain isn’t primed by “this is a test of rationality”.
Some sub-components can be reliably measured—such as calibration. Any others?
My thought on hearing the proposal was that it would be impractically difficult, but on further consideration I suspect it would be much easier than creating a reliable test for intelligence. With proper effort, we should at least be able to beat the standard set by IQ tests
Battleground God
I got through unscathed, but I suspect most atheists would. I don’t think that test is very good at discerning rationality beyond the “knows about logic” and “doesn’t believe in God”.
FWIW, I started it with “God Exists” as true and also got through it unscathed. But you’re right, it doesn’t seem to try very hard presenting atheists with pitfalls and traps. I was at least expecting some kind of foul trickery about the Big Bang.
I would expect it to be extremely hard. I do not limit the scope of these tests to simple paper question and answer. I would be ok if an MRI was needed, for the teste not to know the test was about rationality, if it required a group of well trained actors to fool the teste, or a virtual simulation(within todays technology). Of course tests that can be used/verified immediately by users are preferable.
In the end the techniques required may be prohibitive, but at least it would make known what techniques/technologies to watch/promote for the future.