Today, I finally took a racial/sexual Implicit Association Test.
I had always more or less accepted that it was, if not perfect, at least a fairly meaningful indicator of some sort of bias in the testing population. Now, I’m rather less confident in that conclusion.
According to the test, in terms of positive associations, I rank black women above black men above white women above white men. I do not think this is accurate.
Obviously, this is an atypical result, but I believe that I received it due to confounding factors which prevented the test from being an accurate reflection of my associations are likely to affect a large proportion of the testing population.
First, the most significant factor in how successful I was in correctly associating words and faces was simply practice. I made more mistakes in the first phase than the second phase, and more in the second than the third, etc. I believe that my test could have showed significantly different results simply by re-ordering the phases.
Second, I suspect that I was trying harder in the phases where I was matching black faces than white faces. I don’t want to corrupt the test, but I also don’t want it to tell me I’m a racist; would I have been so enthusiastic about making the final phase my most accurate one of all, if it had been matching white male faces rather than black male faces?
Third, I felt that many of the questions on the survey that followed the matching phase were too loaded to properly answer on their own terms. They presented a series of options from “strongly agree” to “strongly disagree,” where I felt that my real answer would most accurately be framed as ADBOC.
If anyone here has access to university resources and would like to collaborate on an experiment which would attempt to discern subjects’ associations while correcting for these faults, please let me know.
Academic research tends to randomize everything that can be randomized, including the orders of the different IAT phases, so your first concern shouldn’t be an issue in published research. (The keyword for this is “order effect.”)
The IAT is one of several different measures of implicit attitudes which are used in research. When taking the IAT it is transparent to the participant what is being tested in each phase, so people could try harder on some trials than on others, but that is not the case with many of the other tests (many use subliminal priming, e.g. flashing either a black man’s face or a white man’s face on the screen for 20ms immediately before showing the stimulus that participants are instructed to respond to). The different measures tend to produce relatively similar results, which suggests that effort doesn’t have that big of an effect (at least for most people). I suspect that this transparency is part of the reason why the IAT has caught on in popular culture—many people taking the test have the experience of it getting harder when they’re doing a “mismatched” pairing; they don’t need to rely solely on the website’s report of their results.
The survey that you took is not part of the IAT. It is probably a separate, explicit measure of attitudes about race and/or gender (do any of these questions look familiar?).
None of those questions were on the survey, but some of the questions on the survey were similar.
The descriptions of the other measures of implicit attitudes given on that page aren’t in-depth enough for me to critique them effectively for methodology. The first question that comes to mind though, is to what extent these tests have been calibrated against associations that we already know about. For example, if people are given implicit association tests which match words with pictures of, say, smiling children with candy versus pictures of people with injuries, how do they score?
Today, I finally took a racial/sexual Implicit Association Test.
I had always more or less accepted that it was, if not perfect, at least a fairly meaningful indicator of some sort of bias in the testing population. Now, I’m rather less confident in that conclusion.
According to the test, in terms of positive associations, I rank black women above black men above white women above white men. I do not think this is accurate.
Obviously, this is an atypical result, but I believe that I received it due to confounding factors which prevented the test from being an accurate reflection of my associations are likely to affect a large proportion of the testing population.
First, the most significant factor in how successful I was in correctly associating words and faces was simply practice. I made more mistakes in the first phase than the second phase, and more in the second than the third, etc. I believe that my test could have showed significantly different results simply by re-ordering the phases.
Second, I suspect that I was trying harder in the phases where I was matching black faces than white faces. I don’t want to corrupt the test, but I also don’t want it to tell me I’m a racist; would I have been so enthusiastic about making the final phase my most accurate one of all, if it had been matching white male faces rather than black male faces?
Third, I felt that many of the questions on the survey that followed the matching phase were too loaded to properly answer on their own terms. They presented a series of options from “strongly agree” to “strongly disagree,” where I felt that my real answer would most accurately be framed as ADBOC.
If anyone here has access to university resources and would like to collaborate on an experiment which would attempt to discern subjects’ associations while correcting for these faults, please let me know.
Academic research tends to randomize everything that can be randomized, including the orders of the different IAT phases, so your first concern shouldn’t be an issue in published research. (The keyword for this is “order effect.”)
The IAT is one of several different measures of implicit attitudes which are used in research. When taking the IAT it is transparent to the participant what is being tested in each phase, so people could try harder on some trials than on others, but that is not the case with many of the other tests (many use subliminal priming, e.g. flashing either a black man’s face or a white man’s face on the screen for 20ms immediately before showing the stimulus that participants are instructed to respond to). The different measures tend to produce relatively similar results, which suggests that effort doesn’t have that big of an effect (at least for most people). I suspect that this transparency is part of the reason why the IAT has caught on in popular culture—many people taking the test have the experience of it getting harder when they’re doing a “mismatched” pairing; they don’t need to rely solely on the website’s report of their results.
The survey that you took is not part of the IAT. It is probably a separate, explicit measure of attitudes about race and/or gender (do any of these questions look familiar?).
None of those questions were on the survey, but some of the questions on the survey were similar.
The descriptions of the other measures of implicit attitudes given on that page aren’t in-depth enough for me to critique them effectively for methodology. The first question that comes to mind though, is to what extent these tests have been calibrated against associations that we already know about. For example, if people are given implicit association tests which match words with pictures of, say, smiling children with candy versus pictures of people with injuries, how do they score?
I haven’t heard of any attempts at comparing implicit association tests to behavior.