I don’t know of any research that has found a wrong direction effect from anchoring, so this may very well be original research on Less Wrong. Nice job.
I think that it could be consistent with existing models of anchoring. There are two theories for why anchoring effects occur, insufficient adjustment and selective accessibility.
Insufficient adjustment: People get their estimate by anchoring and adjustment, as you described. They start at the anchor, realize it’s too high (for example), and adjust downward until they reach a plausible value. But they tend to stop too soon, since there are a range of plausible values and they don’t go all the way to the most plausible value.
Selective accessibility: Answering the higher/lower question makes you bring to mind information that is relevant to that question. This is a biased set of information, since you bring to mind (for example) information related to the population being over 20 million when considering that possibility. Then when you go to make an estimate in response to the second question, that biased set information remains accessible in your mind and contaminates your judgment.
The general consensus among researchers is that both theories are correct—both processes occur. But there is ongoing disagreement among researchers about precisely when each of the two processes takes place. A few years ago the thought was that there are some cases where anchoring & adjustment takes place and others where selective accessibility happens—with questions like the ones you’d asked, anchoring effects would be due to selective accessibility rather than anchoring & adjustment (as discussed here and here on LW). Since then, a 2010 paper by Simmons, LeBoeuf & Nelson has argued that the typical case is for both selective accessibility and anchoring & adjustment to be operating, so either or both could be involved.
I think that your effect here is consistent with the selective accessibility account. If you’re considering whether the population is more or less than 20 million, you might bring to mind reasons why the population could be more than 20 million and reasons why the population could be less than 20 million. That’s a biased set of all the information that you have, even for the initial question, which will contain more information consistent with the wrong direction than you’d get in a representative sample of your information. That explains why you get closer to a 50⁄50 split in people’s answers compared with what you’d get if you just looked at people’s estimates without the anchoring question. Part of the inspiration for the selective accessibility model was previous research on confirmation bias showing that considering a hypothesis makes it seem more likely, and this wrong direction effect seems consistent with that research.
With anchoring & adjustment, I think you could say that this is a case where the anchor value is already within the range of plausible values. It’s not clear what the model says about those cases, but perhaps it could account for the wrong direction effect. If you see the anchor value and think that it’s a plausible answer, but you’re forced to say “higher” or “lower”, then maybe the choice seems kind of arbitrary and ends up closer to a coin flip.
Simmons, J.P., LeBoeuf, R.A., & Nelson, L.D. (2010). The effect of accuracy motivation on anchoring and adjustment: Do people adjust from provided anchors? Journal of Personality and Social Psychology, 99 (6), 917-932. pdf
Thanks for the citation. I am not aware about the ongoing research in the field and have only very rudimentary knowledge of the relevant theories, so each qualified comment is welcome.
I don’t know of any research that has found a wrong direction effect from anchoring, so this may very well be original research on Less Wrong. Nice job.
I think that it could be consistent with existing models of anchoring. There are two theories for why anchoring effects occur, insufficient adjustment and selective accessibility.
Insufficient adjustment: People get their estimate by anchoring and adjustment, as you described. They start at the anchor, realize it’s too high (for example), and adjust downward until they reach a plausible value. But they tend to stop too soon, since there are a range of plausible values and they don’t go all the way to the most plausible value.
Selective accessibility: Answering the higher/lower question makes you bring to mind information that is relevant to that question. This is a biased set of information, since you bring to mind (for example) information related to the population being over 20 million when considering that possibility. Then when you go to make an estimate in response to the second question, that biased set information remains accessible in your mind and contaminates your judgment.
The general consensus among researchers is that both theories are correct—both processes occur. But there is ongoing disagreement among researchers about precisely when each of the two processes takes place. A few years ago the thought was that there are some cases where anchoring & adjustment takes place and others where selective accessibility happens—with questions like the ones you’d asked, anchoring effects would be due to selective accessibility rather than anchoring & adjustment (as discussed here and here on LW). Since then, a 2010 paper by Simmons, LeBoeuf & Nelson has argued that the typical case is for both selective accessibility and anchoring & adjustment to be operating, so either or both could be involved.
I think that your effect here is consistent with the selective accessibility account. If you’re considering whether the population is more or less than 20 million, you might bring to mind reasons why the population could be more than 20 million and reasons why the population could be less than 20 million. That’s a biased set of all the information that you have, even for the initial question, which will contain more information consistent with the wrong direction than you’d get in a representative sample of your information. That explains why you get closer to a 50⁄50 split in people’s answers compared with what you’d get if you just looked at people’s estimates without the anchoring question. Part of the inspiration for the selective accessibility model was previous research on confirmation bias showing that considering a hypothesis makes it seem more likely, and this wrong direction effect seems consistent with that research.
With anchoring & adjustment, I think you could say that this is a case where the anchor value is already within the range of plausible values. It’s not clear what the model says about those cases, but perhaps it could account for the wrong direction effect. If you see the anchor value and think that it’s a plausible answer, but you’re forced to say “higher” or “lower”, then maybe the choice seems kind of arbitrary and ends up closer to a coin flip.
Simmons, J.P., LeBoeuf, R.A., & Nelson, L.D. (2010). The effect of accuracy motivation on anchoring and adjustment: Do people adjust from provided anchors? Journal of Personality and Social Psychology, 99 (6), 917-932. pdf
Thanks for the citation. I am not aware about the ongoing research in the field and have only very rudimentary knowledge of the relevant theories, so each qualified comment is welcome.