Two ideas I got after 5 minutes (by the clock :)) thinking.
If the tests are stressful and mentally (and possibly physically) exhausting, then even if it is still possible to prepare just for the test, it will not be as far from preparing for the “real thing”. So, something like Initiation Ceremony could be done periodically and not just for initiation.
Give the students “stories” and see if they can make heads or tails of them. (How accurately can they guess the omitted details? Can they predict how it continues? Etc.) But, where can you get real stories? An authored story is very bounded in usefulness for this. The idea: we have court cases. A lot of them, in all kind of domains, dating back to centuries. And they are very real, even if it’s distorted (fake evidence, false testimony), it’s done by someone for some concrete reason, which can be analyzed rationally. This might require learning some Law, but even without formal training many non domain-specific cases can be understood with moderate work. And Law is one of the oldest applications of human rationality.
Both of the ideas are mostly applicable to the second use-case: measuring a bunch of students in a school, but not good for comparing schools or designing a standardized “rationality test”.
Two ideas I got after 5 minutes (by the clock :)) thinking.
If the tests are stressful and mentally (and possibly physically) exhausting, then even if it is still possible to prepare just for the test, it will not be as far from preparing for the “real thing”. So, something like Initiation Ceremony could be done periodically and not just for initiation.
Give the students “stories” and see if they can make heads or tails of them. (How accurately can they guess the omitted details? Can they predict how it continues? Etc.) But, where can you get real stories? An authored story is very bounded in usefulness for this.
The idea: we have court cases. A lot of them, in all kind of domains, dating back to centuries. And they are very real, even if it’s distorted (fake evidence, false testimony), it’s done by someone for some concrete reason, which can be analyzed rationally. This might require learning some Law, but even without formal training many non domain-specific cases can be understood with moderate work. And Law is one of the oldest applications of human rationality.
Both of the ideas are mostly applicable to the second use-case: measuring a bunch of students in a school, but not good for comparing schools or designing a standardized “rationality test”.