The situation described in Pascal’s mugging is OOD (out-of-distribution) for human values. Human values have not been trained/tested on scenarios with tiny probabilites of vast utilities.
What answer does a system that goes OOD give us? It doesn’t matter, we are not supposed to use a system in OOD context.
Naively extrapolating human values too far is not permitted.
Giving an arbitrary/random answer is not permitted.
But we need to make some sort of decision, and we nothing but our values to guide us.
But out values are not defined for the decision we are trying to make.
And we are not allowed to define our values arbitrarily.
I think the answer is really complex, and involves something like “taking all our values and meta-values in account, what is the least arbitrary way we can extend our value system into the space in which we are trying to make a decision”
So, my answer to Pascal’s mugging is: human values are probably not yet ready to answer questions like that, at least not in a consistent manner.
Pascal’s Mugging isn’t OOD. It’s very much in-distribution for human beings historically—there is always a scammer waiting on a street corner offering a product that gives you extremely high utility, at very low probability, of course (imagine a tonic that claims to cure smallpox).
Imagine that I, a Lesswrong forum user, claimed to be from outside the simulation and capable of offering you infinite utility (I’m from a universe where that’s possible) in exchange for a rare Pepe. That’s not a hypothetical offer in a thought experiment. I just did. You had to make a decision when you decided to ignore it. You had to incorporate your values into your decision. You have to do so any time you ignore a scammer, Pascal or otherwise. That’s a revealed preference. And humanity as a whole is remarkably consistent in recognizing its human values to ignore Pascal’s mugger. If Pascal’s mugger is OOD for human values, then anything claiming to give infinite/extremely high utility is also OOD for human values, which depending on your cutoff definitely includes Abrahamic religions and may include the industrial revolution. But those aren’t OOD. They’re a part of life.
I don’t think the commenter is saying that muggings and charlatans are out of distribution for humans. I think he is saying that actual, genuine high utility+low probability decisions are unlikely to occur naturally. Your example isn’t a counterexample because it’s not true and you made it up.
No, I was actually Pascal’s Mugging him. And you. Show them Pepes, mister. Pascal’s Mugger is defined as “a guy who is almost certainly lying extorts you for rare Pepes by promising extreme utility values”. Just by making the offer, I become a very real instance of Pascal’s Mugger.
Similar offers happen every day. Every time a Mormon knocks on your door (or a JW if you’re a Mormon), you have to reason about extreme utility values. And of course “genuine high utility+low probability decisions are unlikely to occur naturally”! They are by definition.
Hmm. You are absolutely right, I didn’t think of all these examples.
Let me rephrase: I think probabilitites on the order of 1/(3^^^3) are OOD for expected utility calculations. We mostly don’t care about expected utility for probabilities that small.
Pascal’s mugging is bucketed either into “this is a scam” or “lottery ticket” by human values. And that is fine, unless this results in a contradiction with some of our other values. But I don’t think it does.
Abrahamic religions
Extremely high utility yes, extremely low probability no. Usually the idea is that you can get the “infinite” reward though hard work, dedication, belief, etc.
The situation described in Pascal’s mugging is OOD (out-of-distribution) for human values. Human values have not been trained/tested on scenarios with tiny probabilites of vast utilities.
What answer does a system that goes OOD give us? It doesn’t matter, we are not supposed to use a system in OOD context.
Naively extrapolating human values too far is not permitted.
Giving an arbitrary/random answer is not permitted.
But we need to make some sort of decision, and we nothing but our values to guide us.
But out values are not defined for the decision we are trying to make.
And we are not allowed to define our values arbitrarily.
I think the answer is really complex, and involves something like “taking all our values and meta-values in account, what is the least arbitrary way we can extend our value system into the space in which we are trying to make a decision”
So, my answer to Pascal’s mugging is: human values are probably not yet ready to answer questions like that, at least not in a consistent manner.
Pascal’s Mugging isn’t OOD. It’s very much in-distribution for human beings historically—there is always a scammer waiting on a street corner offering a product that gives you extremely high utility, at very low probability, of course (imagine a tonic that claims to cure smallpox).
Imagine that I, a Lesswrong forum user, claimed to be from outside the simulation and capable of offering you infinite utility (I’m from a universe where that’s possible) in exchange for a rare Pepe. That’s not a hypothetical offer in a thought experiment. I just did. You had to make a decision when you decided to ignore it. You had to incorporate your values into your decision. You have to do so any time you ignore a scammer, Pascal or otherwise. That’s a revealed preference. And humanity as a whole is remarkably consistent in recognizing its human values to ignore Pascal’s mugger. If Pascal’s mugger is OOD for human values, then anything claiming to give infinite/extremely high utility is also OOD for human values, which depending on your cutoff definitely includes Abrahamic religions and may include the industrial revolution. But those aren’t OOD. They’re a part of life.
I don’t think the commenter is saying that muggings and charlatans are out of distribution for humans. I think he is saying that actual, genuine high utility+low probability decisions are unlikely to occur naturally. Your example isn’t a counterexample because it’s not true and you made it up.
No, I was actually Pascal’s Mugging him. And you. Show them Pepes, mister. Pascal’s Mugger is defined as “a guy who is almost certainly lying extorts you for rare Pepes by promising extreme utility values”. Just by making the offer, I become a very real instance of Pascal’s Mugger.
Similar offers happen every day. Every time a Mormon knocks on your door (or a JW if you’re a Mormon), you have to reason about extreme utility values. And of course “genuine high utility+low probability decisions are unlikely to occur naturally”! They are by definition.
Hmm. You are absolutely right, I didn’t think of all these examples.
Let me rephrase:
I think probabilitites on the order of 1/(3^^^3) are OOD for expected utility calculations.
We mostly don’t care about expected utility for probabilities that small.
Pascal’s mugging is bucketed either into “this is a scam” or “lottery ticket” by human values. And that is fine, unless this results in a contradiction with some of our other values. But I don’t think it does.
Extremely high utility yes, extremely low probability no. Usually the idea is that you can get the “infinite” reward though hard work, dedication, belief, etc.