Over the past few years, we have discreetly approached colleagues faced with a choice between job offers, and asked them to estimate the probability that they will choose one job over another. The average confidence in the predicted choice was a modest 66%, but only 1 of the 24 respondents chose the option to which he or she initially assigned a lower probability, yielding an overall accuracy rate of 96%.
—Dale Griffin and Amos Tversky
Eliezer then notes that this radically changed the way he thought:
When I first read the words above—on August 1st, 2003, at around 3 o’clock in the afternoon—it changed the way I thought. I realized that once I could guess what my answer would be—once I could assign a higher probability to deciding one way than other—then I had, in all probability, already decided. We change our minds less often than we think. And most of the time we become able to guess what our answer will be within half a second of hearing the question.
[...]
But we change our minds less often—much less often—than we think.
But a) this seems like it’s pre-replication crisis, b) regardless, a sample size of 24 is not nearly high enough for me to be very confident in this.
“How often people change their mind” seems like a fairly important question. Anyone know of further work in similar space here? Ideally asking the question from a few different angles.
This question is very sensitive to reference classes and definitions. I change my estimates of my future choices very very often, but the vast majority of my decisions are too trivial to notice that I’m doing so. Yesterday at breakfast I thought I’d probably have tacos for dinner. I didn’t.
For decisions that seem more important, I spend more time on them, and ALSO probably change my mind less often than I intend to. The job-change example is a good one: I usually know what I want after the first few conversations, but I intentionally force myself to consider other alternatives and collect data to be more confident in the decision. Part of my tactics for doing that research and consideration is to understate the chance that I’ll pick the leading option (semi-intentionally; it’s motivated by wanting to make a more reasoned decision in the future, not an honest neutral prediction, but I’d admit that if pressed).
I don’t change my overall values or relationship tenets very often at all, but I do change priorities and activities a whole lot, and am often surprised by what seems best at the time, compared to when planning.
I wonder if one can connect this suggested outcome with the old always go with your first intuition on multiple choice questions.
One aspect to getting answers here seems like it would be about how much is our decision-making about existing preconceptions and biases we carry around and how much might be related to things we actually know but do not keep front of mind or articulate to ourselves while thinking.
“Always go with your first intuition on multiple choice” reflects advice that’s specifically good for students who are anxious because they’re taking a test. The student will generally select the correct answer (or at least the one that’s most likely to be correct). If they’re somewhat uncertain about it, they’ll then start to feel anxious; this anxiety will build over time, resulting in a more and more pessimistic assessment of how likely they are to be correct, resulting in even more anxiety. This continues until the student is either sufficiently pessimistic to think that the original answer was not the best or else changes it simply to relieve the stress. This happens even though no new information has been received, implying said change is unlikely to be correlated with correctness and more likely simply reflects a failure of human psychology.
In short, a test is an especially bad test case (pun fully intended) for this because the amount of bias being introduced increases over time with anxiety, rather than decreasing.
It also might be advice for timed tests—even if the probability of the getting the question right increases after being scrutinized more closely, doing that before answering all the questions might not be the strategy with the highest expected score—if the increase in time taken doesn’t result in an increase in expected score greater than answering an additional question, then opportunity cost may be too high.*
*This may rely on all questions having equal value.
[Question] Do We Change Our Minds Less Often Than We Think?
In “We Change Our Minds Less Often Than We Think”, Eliezer quotes a study:
Eliezer then notes that this radically changed the way he thought:
But a) this seems like it’s pre-replication crisis, b) regardless, a sample size of 24 is not nearly high enough for me to be very confident in this.
“How often people change their mind” seems like a fairly important question. Anyone know of further work in similar space here? Ideally asking the question from a few different angles.
This question is very sensitive to reference classes and definitions. I change my estimates of my future choices very very often, but the vast majority of my decisions are too trivial to notice that I’m doing so. Yesterday at breakfast I thought I’d probably have tacos for dinner. I didn’t.
For decisions that seem more important, I spend more time on them, and ALSO probably change my mind less often than I intend to. The job-change example is a good one: I usually know what I want after the first few conversations, but I intentionally force myself to consider other alternatives and collect data to be more confident in the decision. Part of my tactics for doing that research and consideration is to understate the chance that I’ll pick the leading option (semi-intentionally; it’s motivated by wanting to make a more reasoned decision in the future, not an honest neutral prediction, but I’d admit that if pressed).
I don’t change my overall values or relationship tenets very often at all, but I do change priorities and activities a whole lot, and am often surprised by what seems best at the time, compared to when planning.
I wonder if one can connect this suggested outcome with the old always go with your first intuition on multiple choice questions.
One aspect to getting answers here seems like it would be about how much is our decision-making about existing preconceptions and biases we carry around and how much might be related to things we actually know but do not keep front of mind or articulate to ourselves while thinking.
“Always go with your first intuition on multiple choice” reflects advice that’s specifically good for students who are anxious because they’re taking a test. The student will generally select the correct answer (or at least the one that’s most likely to be correct). If they’re somewhat uncertain about it, they’ll then start to feel anxious; this anxiety will build over time, resulting in a more and more pessimistic assessment of how likely they are to be correct, resulting in even more anxiety. This continues until the student is either sufficiently pessimistic to think that the original answer was not the best or else changes it simply to relieve the stress. This happens even though no new information has been received, implying said change is unlikely to be correlated with correctness and more likely simply reflects a failure of human psychology.
In short, a test is an especially bad test case (pun fully intended) for this because the amount of bias being introduced increases over time with anxiety, rather than decreasing.
It also might be advice for timed tests—even if the probability of the getting the question right increases after being scrutinized more closely, doing that before answering all the questions might not be the strategy with the highest expected score—if the increase in time taken doesn’t result in an increase in expected score greater than answering an additional question, then opportunity cost may be too high.*
*This may rely on all questions having equal value.