So, multiple choice questions are easier to hack than other kinds of questions, but if you “guess” the right answer 80% of the time, that’s not really guessing.
It depends how obvious it is that some of the answers are wrong.
It depends how obvious it is that some of the answers are wrong.
Obvious, of course, is a two-place word.
One experience I had that highlighted the importance of background knowledge for me was playing Alexei’s Calibration Game. I had literally no knowledge about the sports questions- after confirming that, I just clicked A every time, and had 50% accuracy. But one of the categories of questions was “which of these postmasters general of the US served first?”, which I was able to get 60-70% accuracy on, just by replacing that with the question “which of these two American names is more old-timey?”
I doubt a recent Chinese immigrant to the US would be able to hit 60% accuracy with that approach, because they don’t have that good a sense of what’s “old-timey” in the US. (And that’s with only two options!) I’ve seen a handful of prep tests for subjects I know very little about, and my guessing accuracy there is close to chance.
Another way to put this: with multiple choice tests, reversed stupidity is intelligence, and that’s a failing of the test. But identifying stupidity is a skill that requires some expertise.
It depends how obvious it is that some of the answers are wrong.
Obvious, of course, is a two-place word.
One experience I had that highlighted the importance of background knowledge for me was playing Alexei’s Calibration Game. I had literally no knowledge about the sports questions- after confirming that, I just clicked A every time, and had 50% accuracy. But one of the categories of questions was “which of these postmasters general of the US served first?”, which I was able to get 60-70% accuracy on, just by replacing that with the question “which of these two American names is more old-timey?”
I doubt a recent Chinese immigrant to the US would be able to hit 60% accuracy with that approach, because they don’t have that good a sense of what’s “old-timey” in the US. (And that’s with only two options!) I’ve seen a handful of prep tests for subjects I know very little about, and my guessing accuracy there is close to chance.
Another way to put this: with multiple choice tests, reversed stupidity is intelligence, and that’s a failing of the test. But identifying stupidity is a skill that requires some expertise.
But the expertise might not be in the area that’s ostensibly being tested.