I labeled an exam question as “tricky” because if you applied the solution method we used in class to solve similar looking problems you would get the wrong answer. But it occurred to me that if the question had been identical to one given in class but I still labeled it as “tricky” the “tricky” label would have been accurate because the trick would have been students thinking that the obvious answer wasn’t correct when indeed it was. So is it always accurate to label a question as “tricky”?
That’s kind of a Hofstadter-esque question. I think the answer is “no”, but the reason why depends on what meta-level you’re looking at: if the label refers only to the object-level question, then it’s straightforwardly true or false; but if you construe it as applying to the entire context of the question including its labeling, then it’s possible to imagine a trick question that’s transparent enough that labeling it as such exposes the trick and stops it from being tricky. It can be a self-fulfilling or a self-defeating prophecy.
“Tricky” means “the other person is operating at a higher level than I am”.
If you answer a question at a lower level than it was posed, you get marked down for failing to level up. If you answer at a higher level than the question was posed at, and the teacher fails to level up, you get marked down for what failing to level up felt like to the teacher—misinterpreting the question, nitpicking, showing off, whatever. The task in an exam is to figure out what level each question is being asked at, and address it on the same level.
I don’t whether it is tricky or not, but it is the sort of thing I think my students would get legitimately annoyed by. While teaching students to be confident of their answers can be important, I’m not sure in this context that would be helped by this.
Normally you are right, but the class is game theory so I would feel justified playing meta tricky games with my students on exams. Here is a past exam question of mine that got posted.
That makes as much sense as having a class about political corruption and requiring that students pass the test by bribing the teacher.
If I taught a class on political corruption I would totally do that if it wouldn’t get me in trouble.
My goal with that question was to confront the students with a real game theory based moral dilemma. Tests are not just for evaluation, but should also be learning exercises.
But there’s a difference between “this is how you do X” and “doing X is appropriate in this situation”. Deciding that because a class is about bribery, you should get your grade in it by bribery, confuses these two things—you’ve given the students an opportunity to use the lessons from the class, but it’s not a situation where most people think you should have an opportunity to use the lessons from the class. If your class was about some field of statistics related to randomness would you insist that your students roll dice to determine their exam score? If your class was about male privilege, would you automatically give all female students a grade one rank lower?
Tests are not just for evaluation, but should also be learning exercises.
While tests can have purposes, such as learning, that are orthogonal to evaluation, that’s different from giving the test an additional purpose that is counterproductive to evaluation.
Also, I’d hate to be the student who had to explain to a prospective employer that the employer should add a percentage point to his GPA when considering him for employment, on the grounds that he scored poorly in your class for reasons unrelated to evaluation.
Assuming the question was known in advance, the obvious solution is for the people who care more about their grades to pay those who care less to circle A while circling B themselves. If they trust each other, they might even be able to do this after-the-fact.
The universalizing answer would be to choose A 51% of the time.
That’s why you should always have some random bits up your sleeve (memorized).
I remember being surprised that a large number of /r/rational commenters had password systems in case they ever invented time-travel or cloning. Anyone who goes to that effort can presumably also memorize 15 or so random bits if they ever need it, and refresh if used.
Time travel passwords are vulnerable to mindreading. If you want a good time travel password, you have to have an algorithm which the time-travelling version of you can calculate, but which can’t be directly read by a mindreader because if he’s reading it right now, he has no time to calculate it. For instance, I can have a time-travel password of “digits 300-310 of the square root of 3”. A time-travelling version of me would know the password, so can compute it, then can tell me the result and I can check it. A mindreader would have to read my mind before the fact or engage in some time travel himself.
Of course, it’s impossible to have a time-travel password immune to all such tricks (maybe the mindreader did read my mind a week ago), but there’s no reason to allow blatant loopholes.
So is it always accurate to label a question as “tricky”?
If every question is tricky, then the label of “tricky” ceases to be meaningful.
But it occurred to me that if the question had been identical to one given in class but I still labeled it as “tricky” the “tricky” label would have been accurate because the trick would have been students thinking that the obvious answer wasn’t correct when indeed it was.
I labeled an exam question as “tricky” because if you applied the solution method we used in class to solve similar looking problems you would get the wrong answer. But it occurred to me that if the question had been identical to one given in class but I still labeled it as “tricky” the “tricky” label would have been accurate because the trick would have been students thinking that the obvious answer wasn’t correct when indeed it was. So is it always accurate to label a question as “tricky”?
That’s kind of a Hofstadter-esque question. I think the answer is “no”, but the reason why depends on what meta-level you’re looking at: if the label refers only to the object-level question, then it’s straightforwardly true or false; but if you construe it as applying to the entire context of the question including its labeling, then it’s possible to imagine a trick question that’s transparent enough that labeling it as such exposes the trick and stops it from being tricky. It can be a self-fulfilling or a self-defeating prophecy.
In other words, it’s turtles all the way down :-)
“Tricky” means “the other person is operating at a higher level than I am”.
If you answer a question at a lower level than it was posed, you get marked down for failing to level up. If you answer at a higher level than the question was posed at, and the teacher fails to level up, you get marked down for what failing to level up felt like to the teacher—misinterpreting the question, nitpicking, showing off, whatever. The task in an exam is to figure out what level each question is being asked at, and address it on the same level.
I don’t whether it is tricky or not, but it is the sort of thing I think my students would get legitimately annoyed by. While teaching students to be confident of their answers can be important, I’m not sure in this context that would be helped by this.
Normally you are right, but the class is game theory so I would feel justified playing meta tricky games with my students on exams. Here is a past exam question of mine that got posted.
That makes as much sense as having a class about political corruption and requiring that students pass the test by bribing the teacher.
Just because the class is about X doesn’t mean that grades in the class should be measured by X.
If I taught a class on political corruption I would totally do that if it wouldn’t get me in trouble.
My goal with that question was to confront the students with a real game theory based moral dilemma. Tests are not just for evaluation, but should also be learning exercises.
But there’s a difference between “this is how you do X” and “doing X is appropriate in this situation”. Deciding that because a class is about bribery, you should get your grade in it by bribery, confuses these two things—you’ve given the students an opportunity to use the lessons from the class, but it’s not a situation where most people think you should have an opportunity to use the lessons from the class. If your class was about some field of statistics related to randomness would you insist that your students roll dice to determine their exam score? If your class was about male privilege, would you automatically give all female students a grade one rank lower?
While tests can have purposes, such as learning, that are orthogonal to evaluation, that’s different from giving the test an additional purpose that is counterproductive to evaluation.
Also, I’d hate to be the student who had to explain to a prospective employer that the employer should add a percentage point to his GPA when considering him for employment, on the grounds that he scored poorly in your class for reasons unrelated to evaluation.
That one is evil.
Assuming the question was known in advance, the obvious solution is for the people who care more about their grades to pay those who care less to circle A while circling B themselves. If they trust each other, they might even be able to do this after-the-fact.
The universalizing answer would be to choose A 51% of the time.
What was the ratio of As?
Does James Miller let his students take d% dice to his tests?
No, but if a student asked I would be tempted to give her extra credit.
That’s why you should always have some random bits up your sleeve (memorized).
I remember being surprised that a large number of /r/rational commenters had password systems in case they ever invented time-travel or cloning. Anyone who goes to that effort can presumably also memorize 15 or so random bits if they ever need it, and refresh if used.
Time travel passwords are vulnerable to mindreading. If you want a good time travel password, you have to have an algorithm which the time-travelling version of you can calculate, but which can’t be directly read by a mindreader because if he’s reading it right now, he has no time to calculate it. For instance, I can have a time-travel password of “digits 300-310 of the square root of 3”. A time-travelling version of me would know the password, so can compute it, then can tell me the result and I can check it. A mindreader would have to read my mind before the fact or engage in some time travel himself.
Of course, it’s impossible to have a time-travel password immune to all such tricks (maybe the mindreader did read my mind a week ago), but there’s no reason to allow blatant loopholes.
It was several years ago and I don’t remember.
If every question is tricky, then the label of “tricky” ceases to be meaningful.
I believe the word you’re looking for is “cruel”.
Only if you do it once (or a few times) and only after they have seen lots of tricky ones by now.
Reminds me of Vizzini’s battle of wits in the Princess Bride X-)