A large and diverse sample of Americans were surveyed by a national polling firm, Polimetrix/YouGov. They collected party affiliation and other demographic information, and gave each subject the CRT. After completing the CRT, one group of subjects was told that it was a measure of how reflective and open-minded someone is, and that a recent study had found that people who believe in climate change tend to score better than those who doubt climate change. They were then asked asked whether they thought it was a valid measure of how reflective and open-minded someone is. Another group was given the opposite information (climate change doubters scored better), and there was a control condition.
Results: Democrats were more likely to agree that the test was valid if they were told that climate change believers scored better (vs. if climate change doubters scored better), and Republicans showed the opposite pattern. That was their measure of motivated reasoning. Further, 1) the magnitude of the gap was the same for Democrats as it was for Republicans (they were pretty much perfect mirror images of each other), and 2) the magnitude of the gap was bigger for people who scored higher on the CRT than for those who scored lower (and this was the same for Democrats & Republicans). Look at the graph here.
Additional Results: 1) (As you can see on the graph) Among low-CRT people (R or D), hearing that climate change believers did better had a large impact on opinions about the validity of the test (vs. control condition) but hearing that climate change doubters did better had almost no impact; among high-CRT people both pieces of information had similarly large impacts on opinions about validity. 2) There was also a slight trend for Republicans to score higher than Democrats on the CRT; this correlation was much smaller than the correlation between the CRT and other demographic variables.
Here is the exact text of (one version of) the experimental manipulation:
Psychologists believe the questions you have just answered measure how reflective and open-minded someone is. In one recent study, a researcher found that people who accept evidence of climate change tend to get more answers correct than those who reject evidence of climate change. If the test is a valid way to measure open-mindedness, that finding would imply that those who believe climate change is happening are more open-minded than those who are skeptical that climate change is happening.
How strongly do you agree or disagree with this statement? I think the word-problem test I just took supplies good evidence of how reflective and open-minded someone is.
Here’s the design of the study.
A large and diverse sample of Americans were surveyed by a national polling firm, Polimetrix/YouGov. They collected party affiliation and other demographic information, and gave each subject the CRT. After completing the CRT, one group of subjects was told that it was a measure of how reflective and open-minded someone is, and that a recent study had found that people who believe in climate change tend to score better than those who doubt climate change. They were then asked asked whether they thought it was a valid measure of how reflective and open-minded someone is. Another group was given the opposite information (climate change doubters scored better), and there was a control condition.
Results: Democrats were more likely to agree that the test was valid if they were told that climate change believers scored better (vs. if climate change doubters scored better), and Republicans showed the opposite pattern. That was their measure of motivated reasoning. Further, 1) the magnitude of the gap was the same for Democrats as it was for Republicans (they were pretty much perfect mirror images of each other), and 2) the magnitude of the gap was bigger for people who scored higher on the CRT than for those who scored lower (and this was the same for Democrats & Republicans). Look at the graph here.
Additional Results: 1) (As you can see on the graph) Among low-CRT people (R or D), hearing that climate change believers did better had a large impact on opinions about the validity of the test (vs. control condition) but hearing that climate change doubters did better had almost no impact; among high-CRT people both pieces of information had similarly large impacts on opinions about validity. 2) There was also a slight trend for Republicans to score higher than Democrats on the CRT; this correlation was much smaller than the correlation between the CRT and other demographic variables.
Here is the exact text of (one version of) the experimental manipulation: