A month before the paper was assigned a student was talking with their professor and mentioned that the punk band “The Exemplars” was their favorite band and that they would do anything to see them live but that they almost never perform. The day after the paper was due the student comes in and says “I’m sorry I didn’t get the paper in on time, professor, but you won’t believe what happened. The Exemplars, that band I told you about held a one-time-only surprise concert downtown yesterday and I just couldn’t pass up the opportunity to attend. It is true that my excuse isn’t traditionally considered a good one like that of the grieving student, but months ago I signaled to you quite clearly that an Exemplars concert would be the kind of thing that would lead me to turn a paper in late. You excused the grieving student because it is assumed in our culture that a death of a family member will give someone reason not to write a paper. But though we don’t assume most people will not write a paper because of a concert you could have predicted an Exemplars concert would cause me not to write the paper. This excuse won’t be used often by other students since to be successful one has to indicate a stance that would cause them to not write the paper under certain circumstances and then hope those circumstances arise.”
If I were a teacher, I’d ideally allow this (and the sports fan too, for that matter, if I knew he was a big enough fan).
There’s no problem in the ideal form, but I would anticipate a lot of problems in real life—other students not understanding exactly how much this person loved the Exemplars and misinterpreting it as “You get an exemption for going to a rock concert”, other students trying to convince you they like [thing X] just as much as this guy liked the Exemplars and you having to judge lots of difficult cases to see if their fandom is truly as great as this guy’s is and inevitably getting some of them wrong.
If I really knew what to do, this post would have been titled “Eight Short Studies On Excuses, Plus The Answers To Them”
A potential solution for appeasing other students and preventing them from faking Sports Fandom—while still accommodating a Sports Fan’s reasonable situation—is to give the Sports Fan an extra assignment to complete. This would dissuade other students from turning in their paper late (because they would want to avoid having to do extra work), but would satisfy the Sports Fan since they would do anything to be able to see their team, band, etc.
The teacher would still have to have strict guidelines for this accommodation: 1) the request would have to be deemed reasonable 2) the assignment couldn’t be too easy or many students would take advantage of it 3) The extension for the original assignment couldn’t be too accommodating, just long enough to give the student the time they lost from attending the event But this could be a conceivable solution to this problem.
“If it is true that you would really do anything to see them perform, that implies that the performance is worth at least +100 utility to you, to make up for the loss of missing the essay. Therefore, I will allow you to turn it in, but only for 75% credit, disincentivizing lying about your true preferences but still preserving most of the mutual utility.”
You know, most of these problems can be avoided if you accept assignments late and have most of the course grade depend on one or more large individual projects that must be turned in “sometime before the end of the semester”. The simplifying assumption is that a student who doesn’t try to learn anything deserves to remain ignorant—it’s nice how neatly this works out.
I give my students a recommended due date, but allow them to turn in any assignment at any time before Finals Week without penalty, for any reason. It works better than you might think.
(I wish that I was allowed to insist that everybody do an individual project. There is so much cheating it’s a bit ridiculous, and the worst part is that the students who cheat are just good enough at covering their tracks that I can’t find absolutely definitive evidence. This is especially common among Chinese students, for some reason. Again, I figure that if people harm their own learning by cheating on assignments, then their actions are self-punishing.)
In “Predictably Irrational”, they mention a study (don’t have the book with me, can’t be more specific) where a teacher assigns three projects, due at various points throughout the year. The students do relatively well.
In another class the same teacher assigns three projects which can be turned in whenever the students want. The students in this class do quite a bit worse than the first students because they’re procrastinating and do all three projects the last week.
In yet another condition, students are allowed to turn in projects at any time during the year, but they’re also allowed to voluntarily pre-commit to a due date at the beginning of the year (they fail the project if they don’t have it in by their own due date). In this condition, the students who pre-committed did as well as the students in the first class, and the students who didn’t pre-commit did as poorly as the students in the second class.
Based on that study, I predict your students would do better if you gave them assigned due dates.
I do give them assigned due dates, actually; I just make it clear to everyone that I’ll accept arbitrarily late work without penalty. I make a show of collecting papers on the date they’re due. About 80--90% of the students turn in their work on the due date.
Adding to the incentive scheme here is the fact that the classwork builds on earlier work, so if they just procrastinate for a few weeks, they’ll be in a pretty dire situation. If they have to go see a funeral or a concert, or they’ve been really busy some week with exams or a big project for another class, or whatever, then my method gives them enough flexibility for that, but the great majority of people do not procrastinate the way you might predict.
I guess you could call this system “obviously fictitious due dates”. Everybody knows that the due dates are made of handwaving and rainbows, but that doesn’t make them much less effective. And this way I don’t have to listen to any damn excuses, and my students don’t have to think of any; I just smile and recommend catching up before next week.
About 80--90% of the students turn in their work on the due date.
Of the 10-20% who do not turn in their work on the due date, how many eventually turn it in, and how does their quality of work compare to those who turned it in “on time”?
It would be interesting to see a repeat of the experiment Yvain mentions, with another class added using your “obviously fictitious due dates”.
Of the 10-20% who do not turn in their work on the due date, how many eventually turn it in, and how does their quality of work compare to those who turned it in “on time”?
I would say, about 2⁄3 of the people who miss the due date turn in their work a little late. Of those who don’t, a handful will turn in a batch of late papers at the end of the semester, which is a pain in the ass, but at least they did it. And then there are the people who just give up on the class but don’t actually drop it. They make up the bulk of the people who don’t turn in their work ever.
The quality of the late papers is, on average, lower than the quality of the on-time papers. This makes sense; the more diligent students would tend to do better work and get it in on time.
It would be interesting to see a repeat of the experiment Yvain mentions, with another class added using your “obviously fictitious due dates”.
One confounding factor is that the study Yvain mentioned has only three assignments, whereas I have a larger number that are due more frequently. I suspect that my method might not work quite as well with a smaller number of assignments.
The quality of the late papers is, on average, lower than the quality of the on-time papers. This makes sense; the more diligent students would tend to do better work and get it in on time.
Do you have a method for disentangling any negative bias you might have towards late papers (because they are ‘a pain in the ass’) from your quality judgements? I imagine the degree to which completely objective quality measurements are possible is a function of what subject you are teaching.
I hadn’t thought of that, but it turns out that yes, I do! I handle the grading in two phases; in phase one I assign grades, and in phase two I enter them into the grade book. The pain-in-the-ass aspect comes mostly in phase two. I don’t really mind grading late papers; it’s dealing with the school’s broken-ass computer system, Blackboard, that is really aggravating when I’m entering late grades. Or any grades, really, but late grades are worse.
Therefore the grades I assign should be fairly disentangled from any annoyance I may feel later in phase two.
(By the way, nice job thinking up that criticism. The ability to habitually see possible cognitive bias in everyday life is important, and needs to be lauded more.)
(Also, if you happen to be an educational institution, never use Blackboard. It’s the worst thing since BonziBuddy.)
I’m not denying that Blackboard sucks—my university uses it, too—but I’ve not had too much trouble entering grades by the download-edit-upload method. The download and upload options are in the “Grade Center” under “Manage”, and the file is tab-delimited (or some alternate option I have already forgotten about) and editable in Excel.
I’m not quite as flexible as you; my rule is that you can turn it in late (with no penalty) until the day that I return the graded work (which I will delay upon request). I put in this restriction only to avoid the big pile of late papers at the end of the term, which seems to be less of a hassle for you than it is for me. I get similar results.
And yet, there are always a few students who just do not grasp how it works. I get ‘I’m sorry that I missed class yesterday; can I still turn the assignment in today?’ a lot. So ‘obviously fictitious due dates’ is sadly an exaggeration.
Ariely, D., & Wertenbroch, K. (2002). Procrastination, deadlines, and
performance: Self-control and precommitment. Psychological
Science, 13, 219–224. pdf
I should be more clear about what I’m saying. There’s a certain level of evidence needed to prove cheating to The Authorities so that it can be dealt with through the official academic honesty processes of the university. That’s a hard standard to meet, since in this sort of thing you really don’t want false positives. However, I can be almost certain about most instances of cheating, and that’s enough to get a solid idea of how much is going on and who’s doing it most.
I have plenty of evidence, of course, just not enough to convict anybody of academic dishonesty. Except the guy whose idea of “writing a report” was to copy and paste from Wikipedia and hope I didn’t notice. That was weird.
I have plenty of evidence, of course, just not enough to convict anybody of academic dishonesty. Except the guy whose idea of “writing a report” was to copy and paste from Wikipedia and hope I didn’t notice. That was weird.
I need to share this anecdote now… a friend of mine who shall remain nameless was teaching a history class and asked for papers on the War of 1812. One student copied the entry from Uncyclopedia. And showed no signs of it having been a joke. And didn’t understand what she did wrong after it was explained.
The paper explained how one of the major powers in the war was Antarctica, and dolphins carrying bombs helped the United States defend against killer penguins. So yeah.
I’m idly interested (by which I mean I have no use for it, I’m just curious) what some other heuristics are for “obvious cheating.” Mismatch with the student’s apparent understanding of the topic? Or their writing style?
I was accused of copying a paper for, of all things, an economics class, in high school. I think what got me out of it was the completely genuine look of astonishment on my face—I had not, in fact, copied it, and had never gotten the accusation before about anything. To this day I wonder why my teacher thought I had.
Let’s see if I can list a few heuristics for cheating:
Mismatch between the writing style in different parts of the paper. If some paragraphs are poorly-punctuated and ungrammatical, and other parts are written in very formal academic language, that’s a sign that the paper may have been made by copying and pasting from other people’s writings, then filling in the cracks with their own writing.
Formal academic language is a very weak warning sign. It means you should try typing some statistically unlikely phrases into Google, just in case.
Sometimes people who are asked to summarize some assigned reading will do so by copying and pasting directly from it, and changing a bit of the wording around. This is pretty easy to detect if you’ve read it recently as well.
If a completely incompetent student suddenly turns in top quality answers, it’s unlikely that this is due to him just getting his act together.
I have no idea why your teacher thought you copied a paper for your economics class, but those are some heuristics that I’ve learned.
Thanks! I don’t remember much about how I was doing in the class otherwise, so I’m not sure either. Likely possibilities are formal language (I had turned my brain into “paper mode”) or the last one about incompetence (I was not so big on doing homework in high school, so whether I appeared competent would depend on how much I’d been participating in class).
One aspect that complicates the situation with the sports and music fans is an unwillingness to kick people when they are down.
The grieving student, or a student who was sick (even slightly sick, could have still got the report in but at a utility cost) are both people for whom we feel bad.
In contrast your example of the music fan is different. Or the example of a student who says “Last night a Billionaire’s experimental utility AI calculated that giving me a surprise trip to space in a rocket would be worth 10^7 utility points. So I missed the deadline while I was in orbit.” [You can add extra awesome to the example however you like]. In this case we are not at all surprised they missed the deadline, but maybe would be happy to punish, on the basis “meh, the 10^7 utility points you got yesterday aren’t going to be scratched by the −100 for failing this course”.
Would it be reasonable to apply the logical arguments considering religions here? (Music) fandom seems like a relatively new concept—unlike the well-established act of mourning the dead. It seems to come down to whether you put more emphasis on strong personal but pre-established feelings versus known well-established concepts.
You could argue a person using the death of their mother might have no emotional attachment to their mother but are still using her death as an excuse. While this is morally questionable, it still does not set a bad precedent for an exemption on the level of a population. While on the other hand, accepting the excuse of pre-established niche fandom seems to opens up a potentially very large list of exemptions—e.g. I convince the teacher that I’m a real pastafarian at the start of the year with the intent of using this as an excuse later for some set up unique event. There is no way for the teacher to tell the difference between this and pre-established fandom. A dying mother, however, seems to be a largely unpredictable event and the possibility of intentional abuse seems to be much lower.
But doesn’t this make precommitting have a positive expected utility to students, so students would precommit to whatever they thought was most likely to happen and the teacher would still expect more late papers from having this policy.
Well they can’t pick circumstances that are actually likely to come about. If such circumstances can be foreseen the professor will have expected the student to finish the paper earlier just in case.The more likely the event the the more likely the professor is to make that determination and not accept the excuse. Presumably there is some ideal rate of unlikelihood that satisfies the professor’s utility function.
The Music Fan
A month before the paper was assigned a student was talking with their professor and mentioned that the punk band “The Exemplars” was their favorite band and that they would do anything to see them live but that they almost never perform. The day after the paper was due the student comes in and says “I’m sorry I didn’t get the paper in on time, professor, but you won’t believe what happened. The Exemplars, that band I told you about held a one-time-only surprise concert downtown yesterday and I just couldn’t pass up the opportunity to attend. It is true that my excuse isn’t traditionally considered a good one like that of the grieving student, but months ago I signaled to you quite clearly that an Exemplars concert would be the kind of thing that would lead me to turn a paper in late. You excused the grieving student because it is assumed in our culture that a death of a family member will give someone reason not to write a paper. But though we don’t assume most people will not write a paper because of a concert you could have predicted an Exemplars concert would cause me not to write the paper. This excuse won’t be used often by other students since to be successful one has to indicate a stance that would cause them to not write the paper under certain circumstances and then hope those circumstances arise.”
If I were a teacher, I’d ideally allow this (and the sports fan too, for that matter, if I knew he was a big enough fan).
There’s no problem in the ideal form, but I would anticipate a lot of problems in real life—other students not understanding exactly how much this person loved the Exemplars and misinterpreting it as “You get an exemption for going to a rock concert”, other students trying to convince you they like [thing X] just as much as this guy liked the Exemplars and you having to judge lots of difficult cases to see if their fandom is truly as great as this guy’s is and inevitably getting some of them wrong.
If I really knew what to do, this post would have been titled “Eight Short Studies On Excuses, Plus The Answers To Them”
A potential solution for appeasing other students and preventing them from faking Sports Fandom—while still accommodating a Sports Fan’s reasonable situation—is to give the Sports Fan an extra assignment to complete. This would dissuade other students from turning in their paper late (because they would want to avoid having to do extra work), but would satisfy the Sports Fan since they would do anything to be able to see their team, band, etc.
The teacher would still have to have strict guidelines for this accommodation: 1) the request would have to be deemed reasonable 2) the assignment couldn’t be too easy or many students would take advantage of it 3) The extension for the original assignment couldn’t be too accommodating, just long enough to give the student the time they lost from attending the event But this could be a conceivable solution to this problem.
“If it is true that you would really do anything to see them perform, that implies that the performance is worth at least +100 utility to you, to make up for the loss of missing the essay. Therefore, I will allow you to turn it in, but only for 75% credit, disincentivizing lying about your true preferences but still preserving most of the mutual utility.”
You know, most of these problems can be avoided if you accept assignments late and have most of the course grade depend on one or more large individual projects that must be turned in “sometime before the end of the semester”. The simplifying assumption is that a student who doesn’t try to learn anything deserves to remain ignorant—it’s nice how neatly this works out.
I give my students a recommended due date, but allow them to turn in any assignment at any time before Finals Week without penalty, for any reason. It works better than you might think.
(I wish that I was allowed to insist that everybody do an individual project. There is so much cheating it’s a bit ridiculous, and the worst part is that the students who cheat are just good enough at covering their tracks that I can’t find absolutely definitive evidence. This is especially common among Chinese students, for some reason. Again, I figure that if people harm their own learning by cheating on assignments, then their actions are self-punishing.)
In “Predictably Irrational”, they mention a study (don’t have the book with me, can’t be more specific) where a teacher assigns three projects, due at various points throughout the year. The students do relatively well.
In another class the same teacher assigns three projects which can be turned in whenever the students want. The students in this class do quite a bit worse than the first students because they’re procrastinating and do all three projects the last week.
In yet another condition, students are allowed to turn in projects at any time during the year, but they’re also allowed to voluntarily pre-commit to a due date at the beginning of the year (they fail the project if they don’t have it in by their own due date). In this condition, the students who pre-committed did as well as the students in the first class, and the students who didn’t pre-commit did as poorly as the students in the second class.
Based on that study, I predict your students would do better if you gave them assigned due dates.
I do give them assigned due dates, actually; I just make it clear to everyone that I’ll accept arbitrarily late work without penalty. I make a show of collecting papers on the date they’re due. About 80--90% of the students turn in their work on the due date.
Adding to the incentive scheme here is the fact that the classwork builds on earlier work, so if they just procrastinate for a few weeks, they’ll be in a pretty dire situation. If they have to go see a funeral or a concert, or they’ve been really busy some week with exams or a big project for another class, or whatever, then my method gives them enough flexibility for that, but the great majority of people do not procrastinate the way you might predict.
I guess you could call this system “obviously fictitious due dates”. Everybody knows that the due dates are made of handwaving and rainbows, but that doesn’t make them much less effective. And this way I don’t have to listen to any damn excuses, and my students don’t have to think of any; I just smile and recommend catching up before next week.
Of the 10-20% who do not turn in their work on the due date, how many eventually turn it in, and how does their quality of work compare to those who turned it in “on time”?
It would be interesting to see a repeat of the experiment Yvain mentions, with another class added using your “obviously fictitious due dates”.
I would say, about 2⁄3 of the people who miss the due date turn in their work a little late. Of those who don’t, a handful will turn in a batch of late papers at the end of the semester, which is a pain in the ass, but at least they did it. And then there are the people who just give up on the class but don’t actually drop it. They make up the bulk of the people who don’t turn in their work ever.
The quality of the late papers is, on average, lower than the quality of the on-time papers. This makes sense; the more diligent students would tend to do better work and get it in on time.
One confounding factor is that the study Yvain mentioned has only three assignments, whereas I have a larger number that are due more frequently. I suspect that my method might not work quite as well with a smaller number of assignments.
Do you have a method for disentangling any negative bias you might have towards late papers (because they are ‘a pain in the ass’) from your quality judgements? I imagine the degree to which completely objective quality measurements are possible is a function of what subject you are teaching.
I hadn’t thought of that, but it turns out that yes, I do! I handle the grading in two phases; in phase one I assign grades, and in phase two I enter them into the grade book. The pain-in-the-ass aspect comes mostly in phase two. I don’t really mind grading late papers; it’s dealing with the school’s broken-ass computer system, Blackboard, that is really aggravating when I’m entering late grades. Or any grades, really, but late grades are worse.
Therefore the grades I assign should be fairly disentangled from any annoyance I may feel later in phase two.
(By the way, nice job thinking up that criticism. The ability to habitually see possible cognitive bias in everyday life is important, and needs to be lauded more.)
(Also, if you happen to be an educational institution, never use Blackboard. It’s the worst thing since BonziBuddy.)
I’m not denying that Blackboard sucks—my university uses it, too—but I’ve not had too much trouble entering grades by the download-edit-upload method. The download and upload options are in the “Grade Center” under “Manage”, and the file is tab-delimited (or some alternate option I have already forgotten about) and editable in Excel.
I’m not quite as flexible as you; my rule is that you can turn it in late (with no penalty) until the day that I return the graded work (which I will delay upon request). I put in this restriction only to avoid the big pile of late papers at the end of the term, which seems to be less of a hassle for you than it is for me. I get similar results.
And yet, there are always a few students who just do not grasp how it works. I get ‘I’m sorry that I missed class yesterday; can I still turn the assignment in today?’ a lot. So ‘obviously fictitious due dates’ is sadly an exaggeration.
Ariely, D., & Wertenbroch, K. (2002). Procrastination, deadlines, and performance: Self-control and precommitment. Psychological Science, 13, 219–224. pdf
If you can’t find definitive evidence of cheating, what makes you think that a) there is so much of it, and b) especially among Chinese students?
I should be more clear about what I’m saying. There’s a certain level of evidence needed to prove cheating to The Authorities so that it can be dealt with through the official academic honesty processes of the university. That’s a hard standard to meet, since in this sort of thing you really don’t want false positives. However, I can be almost certain about most instances of cheating, and that’s enough to get a solid idea of how much is going on and who’s doing it most.
I have plenty of evidence, of course, just not enough to convict anybody of academic dishonesty. Except the guy whose idea of “writing a report” was to copy and paste from Wikipedia and hope I didn’t notice. That was weird.
I need to share this anecdote now… a friend of mine who shall remain nameless was teaching a history class and asked for papers on the War of 1812. One student copied the entry from Uncyclopedia. And showed no signs of it having been a joke. And didn’t understand what she did wrong after it was explained.
The paper explained how one of the major powers in the war was Antarctica, and dolphins carrying bombs helped the United States defend against killer penguins. So yeah.
One day the classified files will be released, and you’ll be really sorry for having trivialized this theater of the war >:-(
I’m idly interested (by which I mean I have no use for it, I’m just curious) what some other heuristics are for “obvious cheating.” Mismatch with the student’s apparent understanding of the topic? Or their writing style?
I was accused of copying a paper for, of all things, an economics class, in high school. I think what got me out of it was the completely genuine look of astonishment on my face—I had not, in fact, copied it, and had never gotten the accusation before about anything. To this day I wonder why my teacher thought I had.
Let’s see if I can list a few heuristics for cheating:
Mismatch between the writing style in different parts of the paper. If some paragraphs are poorly-punctuated and ungrammatical, and other parts are written in very formal academic language, that’s a sign that the paper may have been made by copying and pasting from other people’s writings, then filling in the cracks with their own writing.
Formal academic language is a very weak warning sign. It means you should try typing some statistically unlikely phrases into Google, just in case.
Sometimes people who are asked to summarize some assigned reading will do so by copying and pasting directly from it, and changing a bit of the wording around. This is pretty easy to detect if you’ve read it recently as well.
If a completely incompetent student suddenly turns in top quality answers, it’s unlikely that this is due to him just getting his act together.
I have no idea why your teacher thought you copied a paper for your economics class, but those are some heuristics that I’ve learned.
Don’t forget to add plagiarism heuristics:
Paper is identical to Student B’s paper
and in CompSci
code-samples in paper are identical to Student B’s code samples (with possibly the comments or variable names altered)
code samples exhibit exactly the same comments/bugs as Student B’s code, even though the code is altered to look somewhat different.
Thanks! I don’t remember much about how I was doing in the class otherwise, so I’m not sure either. Likely possibilities are formal language (I had turned my brain into “paper mode”) or the last one about incompetence (I was not so big on doing homework in high school, so whether I appeared competent would depend on how much I’d been participating in class).
One aspect that complicates the situation with the sports and music fans is an unwillingness to kick people when they are down.
The grieving student, or a student who was sick (even slightly sick, could have still got the report in but at a utility cost) are both people for whom we feel bad.
In contrast your example of the music fan is different. Or the example of a student who says “Last night a Billionaire’s experimental utility AI calculated that giving me a surprise trip to space in a rocket would be worth 10^7 utility points. So I missed the deadline while I was in orbit.” [You can add extra awesome to the example however you like]. In this case we are not at all surprised they missed the deadline, but maybe would be happy to punish, on the basis “meh, the 10^7 utility points you got yesterday aren’t going to be scratched by the −100 for failing this course”.
Would it be reasonable to apply the logical arguments considering religions here? (Music) fandom seems like a relatively new concept—unlike the well-established act of mourning the dead. It seems to come down to whether you put more emphasis on strong personal but pre-established feelings versus known well-established concepts.
You could argue a person using the death of their mother might have no emotional attachment to their mother but are still using her death as an excuse. While this is morally questionable, it still does not set a bad precedent for an exemption on the level of a population. While on the other hand, accepting the excuse of pre-established niche fandom seems to opens up a potentially very large list of exemptions—e.g. I convince the teacher that I’m a real pastafarian at the start of the year with the intent of using this as an excuse later for some set up unique event. There is no way for the teacher to tell the difference between this and pre-established fandom. A dying mother, however, seems to be a largely unpredictable event and the possibility of intentional abuse seems to be much lower.
But doesn’t this make precommitting have a positive expected utility to students, so students would precommit to whatever they thought was most likely to happen and the teacher would still expect more late papers from having this policy.
Well they can’t pick circumstances that are actually likely to come about. If such circumstances can be foreseen the professor will have expected the student to finish the paper earlier just in case.The more likely the event the the more likely the professor is to make that determination and not accept the excuse. Presumably there is some ideal rate of unlikelihood that satisfies the professor’s utility function.
On the other hand, shit happens, and anyone with good sense would have done the paper as soon as they could, just in case.