I was once in a forecasting workshop with ~11 other people where we were given a sheet of paper that asked us to estimate the probability of a future event. We all knew about Fermi modeling, and we were allowed to use the internet to look up relevant facts, so I really felt I was giving my very best guess. But after we turned the papers in, the people running the workshop revealed that there was a number written in the top right corner of each sheet of paper (like where a page number would usually be), and half of us had gotten one number (relatively big, like 50 or something), and the other half a different number (smaller, like 2). I don’t remember the specifics but the effect size (using the term colloquially) was ridiculously huge, where the average of the first group’s guesses was something like an order of magnitude more than the average of the second group’s.
I have been completely baffled by this experience ever since and have no idea what to do with it. When I told my husband about it he was like, “wtf, all 12 of you should be fired.” Maybe he’s right? It just seems impossible??? And yet, this is a workshop they’d run many times, and clearly an outcome they were expecting, and I don’t have reason to believe they lied.
Fuck I’m so confused can anyone make sense of this experience
A 2008 paper found anchoring effects from these kinds of “incidental environmental anchors”, but then a replication of one of its studies with a much larger sample size found no effect (see “9. Influence of incidental anchors on judgment (Critcher & Gilovich, 2008, Study 2)”).
So that at least says something about why the people running your forecasting workshop thought this would have an effect, and provides some entry points into the published research which someone could look into in more depth, but it still leaves it surprising/confusing that there was such a large difference.
I was once in a forecasting workshop with ~11 other people where we were given a sheet of paper that asked us to estimate the probability of a future event. We all knew about Fermi modeling, and we were allowed to use the internet to look up relevant facts, so I really felt I was giving my very best guess. But after we turned the papers in, the people running the workshop revealed that there was a number written in the top right corner of each sheet of paper (like where a page number would usually be), and half of us had gotten one number (relatively big, like 50 or something), and the other half a different number (smaller, like 2). I don’t remember the specifics but the effect size (using the term colloquially) was ridiculously huge, where the average of the first group’s guesses was something like an order of magnitude more than the average of the second group’s.
I have been completely baffled by this experience ever since and have no idea what to do with it. When I told my husband about it he was like, “wtf, all 12 of you should be fired.” Maybe he’s right? It just seems impossible??? And yet, this is a workshop they’d run many times, and clearly an outcome they were expecting, and I don’t have reason to believe they lied.
Fuck I’m so confused can anyone make sense of this experience
A 2008 paper found anchoring effects from these kinds of “incidental environmental anchors”, but then a replication of one of its studies with a much larger sample size found no effect (see “9. Influence of incidental anchors on judgment (Critcher & Gilovich, 2008, Study 2)”).
So that at least says something about why the people running your forecasting workshop thought this would have an effect, and provides some entry points into the published research which someone could look into in more depth, but it still leaves it surprising/confusing that there was such a large difference.