Why can’t we use wrong probabilities in real life?
There are various circumstances where I want to “rotate” between probabilities and utilities in real life, in ways that still prescribe the correct decisions. For example, if I have a startup idea and want to maximize my expected profit, I’d be much more emotionally comfortable with thinking it has a 90% chance of making $1 billion than a 10% chance of making $9 billion. So why can’t we use wrong probabilities in real life?I think there are three major reasons why this doesn’t always work.
Idea: with a bounded world-model, messing with your probabilities changes probabilities of other things.
Idea: It’s just easier to have the correct policy if your probabilities are aligned with frequentist probabilities. Despite the extra degree of freedom, we don’t know of any better algorithm that lets us update towards the correct combination of probability and utility, other than getting each right individually.
You can no longer use evidence the same way.
Suppose you believe your startup is 90% to succeed, rather than the true probability of 10%.
A naive Bayesian update 2:1 towards the startup working now brings you to ~95%, not ~18%. Your model gives a smaller change in expected utility than reality.
To get the correct expected utility, the new probability of your startup succeeding has to be 164%. Something has gone wrong when you estimate a probability of 164%. I think you basically have to say probability of this is upper bounded at 900%, which makes sense because it’s not magic, it’s just shuffling probabilities around
It also seems like you are not actually utility indifferent as between a 90% chance of $1b and a 9% chance of $9b—the former seems far more valuable to me because once you have about ~$10m, the rest is just points on a scoreboard. So to the extent that you are more emotionally comfortable with 90%/$1b I think it’s actually because the expected utility of 90%/$1b is almost 10 times higher than the expected utility of 9%/$9b. And so to the extent you are setting your motivation based on these 2 things, you are importantly fooling yourself here.
TBH, for the equation “Util(90% chance of $1b) = Util(9% chance of $X)”, I don’t think there is any finite X that can solve that equation.
Why can’t we use wrong probabilities in real life?
There are various circumstances where I want to “rotate” between probabilities and utilities in real life, in ways that still prescribe the correct decisions. For example, if I have a startup idea and want to maximize my expected profit, I’d be much more emotionally comfortable with thinking it has a 90% chance of making $1 billion than a 10% chance of making $9 billion. So why can’t we use wrong probabilities in real life?I think there are three major reasons why this doesn’t always work.
Idea: with a bounded world-model, messing with your probabilities changes probabilities of other things.
Idea: It’s just easier to have the correct policy if your probabilities are aligned with frequentist probabilities. Despite the extra degree of freedom, we don’t know of any better algorithm that lets us update towards the correct combination of probability and utility, other than getting each right individually.
You can no longer use evidence the same way.
Suppose you believe your startup is 90% to succeed, rather than the true probability of 10%.
A naive Bayesian update 2:1 towards the startup working now brings you to ~95%, not ~18%. Your model gives a smaller change in expected utility than reality.
To get the correct expected utility, the new probability of your startup succeeding has to be 164%. Something has gone wrong when you estimate a probability of 164%. I think you basically have to say probability of this is upper bounded at 900%, which makes sense because it’s not magic, it’s just shuffling probabilities around
It also seems like you are not actually utility indifferent as between a 90% chance of $1b and a 9% chance of $9b—the former seems far more valuable to me because once you have about ~$10m, the rest is just points on a scoreboard. So to the extent that you are more emotionally comfortable with 90%/$1b I think it’s actually because the expected utility of 90%/$1b is almost 10 times higher than the expected utility of 9%/$9b. And so to the extent you are setting your motivation based on these 2 things, you are importantly fooling yourself here.
TBH, for the equation “Util(90% chance of $1b) = Util(9% chance of $X)”, I don’t think there is any finite X that can solve that equation.