And testing confirmation bias in this fashion is intrinsically deceptive, so I probably have damaged my online reputation as well.
Yes, this test might not have passed a university ethics review—not only did you deceive your subject, which must be done carefully, but he didn’t even know he was participating in an experiment!
Amateur Science -
I do what I must,
because, I can.
For the good of all of us.
Except the ones who were tricked.
But there's no sense crying over all the missed frills,
You just keep on trying until you run out of pills.
And the Science* was fun,
And you get neat posts done
For the people who are, still alive.
* This parody has not been approved by the FDA for any medicinal purposes nor has it been replicated.
This is cute, but I think it would have been worth trying to salvage the rhyme scheme. (Maybe ”… all the missed frills”/”pills” and “And the Science helps most”/”post”?) You also missed a huge opportunity by not starting a line earlier with the words “Amateur science” (sound-alike with Aperture).
Amateur was a major omission, yeah. I don’t think ‘helps most/post’ really works because it diverges too much. ‘Done’ is hard to rhyme with, but checking a rhyming dictionary, I realized there was a perfect rhyme: ‘fun’. As in, ‘And the Science was fun / and you make a neat post’. The rhythms scan, the rhyme is perfect, and it makes sense by reinforcing ‘neat’.
Why is it ethically difficult? (I don’t particularly care whether it passes a university ethics review, I base whether I consider it ethical on my emotional reaction to it and emotional reaction to its consequences)
There is some ethical difficulty in running a study (and performing an intervention) on subjects who don’t even know that they are in a study. In the application for a research position, people thought they were applying for a position, not tested for their responses to stimuli about the use of their work.
Obviously, certain kinds of results are impossible to obtain when you tell the subjects that they are part of an experiment. The value of the results might justify the deception, particularly in cases like these two studies in which the participants did not suffer any significant harm. But it is unrealistic to pretend that no deception occurred, or that deception is not a flag for potential ethical difficulty.
Section 8.02, for starts. And yes, 8.05 and 8.07 provides an exception, but it’s debatable whether it applies here, where harm was actually done to Seth Roberts’ reputation. It’s less debatable that it applies to the researcher study, where apparently no harm was done.
Well, presumably if working harder on their submission was their utility-maximizing choice, they would have done so already sans experimental manipulation; if any more quality time was used up, it probably came at the expense of some other activity...
It looks like I badly misunderstood your comment. When you wrote “the researchers,” I thought that was a coy way of referring to yourself in reference to the two experimental results of which I questioned the ethics.
I’m not arguing for the optimality of compliance with an IRB or other “ethical” guidelines—I’m doubtful they do a reasonable job of creating morally optimal research protocols, and they clearly prevent the discovery of certain interesting or useful results—like your results from these posts that relied on deception. And that doesn’t even account for the compliance costs that I now realize was the point of your comment. Oops
I can’t find it in my comment history right now, but I’ve also brought up the apparent lack of ethical oversight in SI-related experiments before. I think the first time was during the first rationality mini-camp.
Yes, this test might not have passed a university ethics review—not only did you deceive your subject, which must be done carefully, but he didn’t even know he was participating in an experiment!
* This parody has not been approved by the FDA for any medicinal purposes nor has it been replicated.
This is cute, but I think it would have been worth trying to salvage the rhyme scheme. (Maybe ”… all the missed frills”/”pills” and “And the Science helps most”/”post”?) You also missed a huge opportunity by not starting a line earlier with the words “Amateur science” (sound-alike with Aperture).
Amateur was a major omission, yeah. I don’t think ‘helps most/post’ really works because it diverges too much. ‘Done’ is hard to rhyme with, but checking a rhyming dictionary, I realized there was a perfect rhyme: ‘fun’. As in, ‘And the Science was fun / and you make a neat post’. The rhythms scan, the rhyme is perfect, and it makes sense by reinforcing ‘neat’.
I like how this response is just as relevant to your non-IRB approvable study on what stimuli cause more effort.
Please tell me that I not the first person to note the (ethically difficult) deceptiveness of that study, even if the results are interesting.
Why is it ethically difficult? (I don’t particularly care whether it passes a university ethics review, I base whether I consider it ethical on my emotional reaction to it and emotional reaction to its consequences)
There is some ethical difficulty in running a study (and performing an intervention) on subjects who don’t even know that they are in a study. In the application for a research position, people thought they were applying for a position, not tested for their responses to stimuli about the use of their work.
Obviously, certain kinds of results are impossible to obtain when you tell the subjects that they are part of an experiment. The value of the results might justify the deception, particularly in cases like these two studies in which the participants did not suffer any significant harm. But it is unrealistic to pretend that no deception occurred, or that deception is not a flag for potential ethical difficulty.
Section 8.02, for starts. And yes, 8.05 and 8.07 provides an exception, but it’s debatable whether it applies here, where harm was actually done to Seth Roberts’ reputation. It’s less debatable that it applies to the researcher study, where apparently no harm was done.
Arguably, the researchers were harmed inasmuch as they were induced to apply more effort/time than they otherwise would have.
Putting in more work to get another interesting experimental result is a harm to the researcher? On what planet?
Well, presumably if working harder on their submission was their utility-maximizing choice, they would have done so already sans experimental manipulation; if any more quality time was used up, it probably came at the expense of some other activity...
It looks like I badly misunderstood your comment. When you wrote “the researchers,” I thought that was a coy way of referring to yourself in reference to the two experimental results of which I questioned the ethics.
I’m not arguing for the optimality of compliance with an IRB or other “ethical” guidelines—I’m doubtful they do a reasonable job of creating morally optimal research protocols, and they clearly prevent the discovery of certain interesting or useful results—like your results from these posts that relied on deception. And that doesn’t even account for the compliance costs that I now realize was the point of your comment. Oops
I can’t find it in my comment history right now, but I’ve also brought up the apparent lack of ethical oversight in SI-related experiments before. I think the first time was during the first rationality mini-camp.
It would have been passed by the IRB of Gwern U. As long as Gwern is not affiliated with another IRB, that’s all that matters.