They key is probably to test someone without letting them know you are testing them. If I ran a martial arts dojo and wanted to make sure my students were really super badass ninjas, I would give them a convincing looking “test” that included things you would expect to see: strength, speed, form, technique, success in actual matches, etc.
This would have very little weighting in the actual grade, however. The real test would be some sort of surprise fight or fights where the student has no idea that the fight is actually one of the tests. Perhaps he (or she) is followed by the assailant until an opportunity to pick a fight arises.
The main advantage of the surprise test is that it is much hard to game. Imperfect metrics are much more likely to say something meaningful about the student in this surprise situation than if the student knows the test is coming.
When it comes to the rationality dojo, there are numerous normally easy-to-game heuristics that could be used, for example:
how susceptible the student is to group-think
what they do in some sort of strenuous situation (e.g., do they blow up the Huygens?) The situation must seem real to them.
are they willing to bet their beliefs even when no one important will notice?
What others can you guys think of?
edit: notice that lists are not working.
edit 2: never mind, editing seemed to fix them.
I doubt that it would be practical to analyze all of the information and get a single number as a measure of the student’s rationality. At the top of all of these tests would have to be someone whose judgment on matters of rationality can be trusted. This may be the most difficult part
Also note that this form of testing would probably be expensive.
They key is probably to test someone without letting them know you are testing them. If I ran a martial arts dojo and wanted to make sure my students were really super badass ninjas, I would give them a convincing looking “test” that included things you would expect to see: strength, speed, form, technique, success in actual matches, etc.
This would have very little weighting in the actual grade, however. The real test would be some sort of surprise fight or fights where the student has no idea that the fight is actually one of the tests. Perhaps he (or she) is followed by the assailant until an opportunity to pick a fight arises.
The main advantage of the surprise test is that it is much hard to game. Imperfect metrics are much more likely to say something meaningful about the student in this surprise situation than if the student knows the test is coming.
When it comes to the rationality dojo, there are numerous normally easy-to-game heuristics that could be used, for example:
how susceptible the student is to group-think
what they do in some sort of strenuous situation (e.g., do they blow up the Huygens?) The situation must seem real to them.
are they willing to bet their beliefs even when no one important will notice?
What others can you guys think of?
edit: notice that lists are not working. edit 2: never mind, editing seemed to fix them.
I doubt that it would be practical to analyze all of the information and get a single number as a measure of the student’s rationality. At the top of all of these tests would have to be someone whose judgment on matters of rationality can be trusted. This may be the most difficult part
Also note that this form of testing would probably be expensive.
See artemis fowl and the butler training.