I feel like I vaguely recall reading somewhere that some sort of california canvassing to promote gay rights experiment either didn’t replicate or turned out to be outright fraud. Wish I could remember the details. It wasn’t the experiment you are talking about though hopefully?
I just started the audiobook myself, and in the part I’m up to the author mentioned that there was a study of deep canvassing that was very bad and got retracted, but then later, there was a different group of scientists who studied deep canvassing, more on which later in the book. (I haven’t gotten to the “later in the book” yet.)
“If a fraudulent paper says the sky is blue, that doesn’t mean it’s green” :)
UPDATE: Yeah, my current impression is that the first study was just fabricated data. It wasn’t that the data showed bad results so he massaged it, more like he never bothered to get data in the first place. The second study found impressive results (supposedly—I didn’t scrutinize the methodology or anything) and I don’t think the first study should cast doubt on the second study.
Thanks for the summary. Yes, David addresses this in the book. There was an unfortunately fraudulent paper published due to (IIRC) the actions of a grad student, but the professors involved retracted the original paper and later research reaffirmed the approach did work.
I read this. This is about the first study, which was retracted. However, a second, carefully monitored and reviewed study found most of the same results, including the remarkably high effect size of one in ten people appearing to completely drop their prejudice toward homosexuals after the ten-minute intervention.
Yes beware the one study. But in the absence of data, small amounts are worth a good deal, and careful reasoning from other evidence is worth even more.
My reasoning from indirect data and personal experience are in line with this one study. The 900 studies on how minds don’t change are almost all about impersonal, data-and-argument based approaches.
Emotions affect how we make and change beliefs. You can’t force someone to change their mind, but they can and do change their mind when they happen to think through an issue without being emotionally motivated to keep their current belief.
As mentioned above, David addresses this in the book. There was an unfortunately fraudulent paper published due to (IIRC) the actions of a grad student, but the professors involved retracted the original paper and later research reaffirmed the approach did work.
Great post, will buy the book and take a look!
I feel like I vaguely recall reading somewhere that some sort of california canvassing to promote gay rights experiment either didn’t replicate or turned out to be outright fraud. Wish I could remember the details. It wasn’t the experiment you are talking about though hopefully?
I just started the audiobook myself, and in the part I’m up to the author mentioned that there was a study of deep canvassing that was very bad and got retracted, but then later, there was a different group of scientists who studied deep canvassing, more on which later in the book. (I haven’t gotten to the “later in the book” yet.)
Wikipedia seems to support that story, saying that the first guy was just making up data (see more specifically “When contact changes minds” on wikipedia).
“If a fraudulent paper says the sky is blue, that doesn’t mean it’s green” :)
UPDATE: Yeah, my current impression is that the first study was just fabricated data. It wasn’t that the data showed bad results so he massaged it, more like he never bothered to get data in the first place. The second study found impressive results (supposedly—I didn’t scrutinize the methodology or anything) and I don’t think the first study should cast doubt on the second study.
Thanks for the summary. Yes, David addresses this in the book. There was an unfortunately fraudulent paper published due to (IIRC) the actions of a grad student, but the professors involved retracted the original paper and later research reaffirmed the approach did work.
https://statmodeling.stat.columbia.edu/2015/12/16/lacour-and-green-1-this-american-life-0/
and generally “beware the one of just one study”
I read this. This is about the first study, which was retracted. However, a second, carefully monitored and reviewed study found most of the same results, including the remarkably high effect size of one in ten people appearing to completely drop their prejudice toward homosexuals after the ten-minute intervention.
Yes beware the one study. But in the absence of data, small amounts are worth a good deal, and careful reasoning from other evidence is worth even more.
My reasoning from indirect data and personal experience are in line with this one study. The 900 studies on how minds don’t change are almost all about impersonal, data-and-argument based approaches.
Emotions affect how we make and change beliefs. You can’t force someone to change their mind, but they can and do change their mind when they happen to think through an issue without being emotionally motivated to keep their current belief.
As mentioned above, David addresses this in the book. There was an unfortunately fraudulent paper published due to (IIRC) the actions of a grad student, but the professors involved retracted the original paper and later research reaffirmed the approach did work.