In the sequences, one of the beisutsukai rituals involves having to perform an on-the-fly bayesian update mentally (and then resist being peer pressured into a wrong answer)
Something to do with the planning fallacy
Confirmation bias—see the harry/hermione scene on the train
Something to do with overcomplicated plans/theories
“What grade will you get on this test?” → graded on the accuracy. sort of a calibration-plus-humility question
something to test whether they can notice the “quiet strain” that signals something’s wrong with a theory—maybe successively presented with evidence that counters a theory they’ve formed, and see how long it takes them to figure it out
curiosity, some small thread that’s out of place and they have to tug on instead of accepting the obvious solution. this is sort of like the CRT, but I’m thinking more like they’re given an explicit framework and there’s something tiny out of place, rather than there being a wrong s1 answer and a right s2 answer
“name a time where you believed yourself to be correct with absolute or near-absolute confidence, and describe the process of how you came to realize that you were wrong”
an authority states something wrong, but subtly wrong, see if they can notice the incorrectness and disagree. “Perhaps your conception of rationality is that it is rational to believe the words of the Great Teacher, and the Great Teacher says, “The sky is green,” and you look up at the sky and see blue. If you think, “It may look like the sky is blue, but rationality is to believe the words of the Great Teacher,” you lose a chance to discover your mistake.”
I agree with ChristianKi that avoiding goodharting will be a major challenge.
Edit:
Please describe a future event or belief that you are 100% certain will happen or is correct.
Putting anything down that isn’t a rejection of the question is zero points. This is basic probability theory mixed with ability to think for yourself (the question is framed to imply that there are 100% certainties).
Something along the lines of the CRT
Calibration questions
In the sequences, one of the beisutsukai rituals involves having to perform an on-the-fly bayesian update mentally (and then resist being peer pressured into a wrong answer)
Something to do with the planning fallacy
Confirmation bias—see the harry/hermione scene on the train
Something to do with overcomplicated plans/theories
“What grade will you get on this test?” → graded on the accuracy. sort of a calibration-plus-humility question
something to test whether they can notice the “quiet strain” that signals something’s wrong with a theory—maybe successively presented with evidence that counters a theory they’ve formed, and see how long it takes them to figure it out
curiosity, some small thread that’s out of place and they have to tug on instead of accepting the obvious solution. this is sort of like the CRT, but I’m thinking more like they’re given an explicit framework and there’s something tiny out of place, rather than there being a wrong s1 answer and a right s2 answer
“name a time where you believed yourself to be correct with absolute or near-absolute confidence, and describe the process of how you came to realize that you were wrong”
an authority states something wrong, but subtly wrong, see if they can notice the incorrectness and disagree. “Perhaps your conception of rationality is that it is rational to believe the words of the Great Teacher, and the Great Teacher says, “The sky is green,” and you look up at the sky and see blue. If you think, “It may look like the sky is blue, but rationality is to believe the words of the Great Teacher,” you lose a chance to discover your mistake.”
I agree with ChristianKi that avoiding goodharting will be a major challenge.
Edit: Please describe a future event or belief that you are 100% certain will happen or is correct.
Putting anything down that isn’t a rejection of the question is zero points. This is basic probability theory mixed with ability to think for yourself (the question is framed to imply that there are 100% certainties).