If someone suggests to me that they have the ability to save 3^^^3 lives, and I assign this a 1/3^^^3 probability, and then they open a gap in the sky at billions to one odds, I would conclude that it is still extremely unlikely that they can save 3^^^3 lives. However, it is possible that their original statement is false and yet it would be worth giving them five dollars because they would save a billion lives. Of course, this would require further assumptions on whether people are likely to do things that they have not said they would do, but are weaker versions of things they did say they would do but are not capable of.
Also, I would assign lower probabilities when they claim they could save more people, for reasons that have nothing to do with complexity. For instance, “the more powerful a being is, the less likely he would be interested in five dollars” or :”a fraudster would wish to specify a large number to increase the chance that his fraud succeeds when used on ordinary utility maximizers, so the larger the number, the greater the comparative likelihood that the person is fraudulent”.
the phrase “Pascal’s Mugging” has been completely bastardized to refer to an emotional feeling of being mugged that some people apparently get when a high-stakes charitable proposition is presented to them, regardless of whether it’s supposed to have a low probability.
1) Sometimes what you may actually be seeing is disagreement on whether the hypothesis has a low probability.
2) Some of the arguments against Pascal’s Wager and Pascal’s Mugging don’t depend on the probability. For instance, Pascal’s Wager has the “worshipping the wrong god” problem—what if there’s a god who prefers that he not be worshipped and damns worshippers to Hell? Even if there’s a 99% chance of a god existing, this is still a legitimate objection (unless you want to say there’s a 99% chance specifically of one type of god).
3) In some cases, it may be technically true that there is no low probability involved but there may be some other small number that the size of the benefit is multiplied by. For instance, most people discount events that happen far in the future. A highly beneficial event that happens far in the future would have the benefit multiplied by a very small number when considering discounting.
Of course in cases 2 and 3 that is not technically Pascal’s mugging by the original definition, but I would suggest the definition should be extended to include such cases. Even if not, they should at least be called something that acknowledges the similarity, like “Pascal-like muggings”.
1) It’s been applied to cryonic preservation, fer crying out loud. It’s reasonable to suspect that the probability of that working is low, but anyone who says with current evidence that the probability is beyond astronomically low is being too silly to take seriously.
The benefit of cryonic preservation isn’t astronomically high, though, so you don’t need a probability that is beyond astronomically low. First of all,even an infinitely long life after being revived only has a finite present value, and possibly a very low one, because of discounting. Second, the benefit from cryonics is the benefit you’d gain from being revived after being cryonically preserved, minus the benefit that you’d gain from being revived after not cryonically preserved. (A really advanced society might be able to simulate us. If simulations count as us, simulating us counts as reviving us without the need for cryonic preservation.)
I don’t think that either Pascal’s Wager or Pascal’s Mugging requires a probability that is astronomically low. It just requires that the size of the purported benefit be large enough that it overwhelms the low probability of the event.
Even if not, they should at least be called something that acknowledges the similarity, like “Pascal-like muggings”.
Any similarities are arguments for giving them a maximally different name to avoid confusion, not a similar one. Would the English language really be better if rubies were called diyermands?
Chemistry would not be improved by providing completely different names to chlorate and perchlorate (e.g. chlorate and sneblobs). Also, I think English might be better if rubies were called diyermands. If all of the gemstones were named something that followed a scheme similar to diamonds, that might be an improvement.
I disagree. Communication can be noisy, and if a bit of noise replaces a word with a word in a totally different semantic class the error can be recovered, whereas if it replaces it with a word in the similar class it can’t. See the last paragraph in myl’s comment to this comment.
Humans have the luxury of neither perfect learning nor perfect recall. In general, I find that my ability to learn and ability to recall words are much more limiting generally speaking than noisy communication channels. I think that there are other sources of redundancy in human communication that make noise less of an issue. For example, if I’m not sure if someone said “chlorate” or “perchlorate” often the ambiguity would be obvious, such as if it is clear that they had mumbled so I wasn’t quite sure what they said. In the case of the written word, Chemistry and context provide a model for things which adds as a layer of redundancy, similar to the language model described in the post you linked to.
It would take me at least twice as long to memorize random/unique alternatives to hypochlorite, chlorite, chlorate, perchlorate, multiplied by all the other oxyanion series. It would take me many times as long to memorize unique names for every acetyl compound, although I obviously acknowledge that Chemistry is the best case scenario for my argument and worst case scenario for yours. In the case of philosophy, I still think there are advantages to learning and recall for similar things to be named similarly. Even in the case of “Pascal’s mugging” vs. “Pascal’s wager”, I believe that it is easier to recall and thus easier to have cognition about in part because of the naming connection between the two, despite the fact that these are two different things.
Note that I am not saying I am in favor of calling any particular thing “Pascal-like muggings,” which draws an explicit similarity between the two, all I’m saying is that choosing a “maximally different name to avoid confusion” strikes me as being less ideal, and that if you called it a Jiro’s mugging or something, that would more than enough semantic distance between the ideas.
Chemistry would not be improved by providing completely different names to chlorate and perchlorate (e.g. chlorate and sneblobs).
Okay, thats actually a good example. This caused me to re-think my position. After thinking, I’m still not sure that the analogy is actually valid though.
In chemistry, we have a systemic naming scheme. Systematic name schemes are good, because they let us guess word meanings without having to learn them. In a difficult field which most people learn only as adults if at all, this is a very good thing. I’m no chemist, but if I had to guess the words chlorate and perchlorate to cause confusion sometimes, but that this price is overall worth paying for a systemic naming scheme.
For gemstones, we do not currently have a systematic naming scheme. I’m not entirely sure that bringing one in would be good, there aren’t all that many common gemstones that we’re likely to forget them and frankly if it ain’t broke don’t fix it, but I’m not sure it would be bad either.
What would not be good would be to simply rename rubies to diyermands without changing anything else. This would not only result in misunderstandings, but generate the false impression that rubies and diamonds have something special in common as distinct from Sapphires and Emeralds (I apologise for my ignorance if this is in fact the case).
But at least in the case of gemstones we do not already have a serious problem, I do not know of any major epistemic failures floating around to do with the diamond-ruby distinction.
In the case of Pascal’s mugging, we have a complete epistemic disaster, a very specific very useful term have been turned into a useless bloated red-giant word, laden with piles of negative connotations and no actual meaning beyond ‘offer of lots of utility that I need an excuse to ignore’.
I know of almost nobody who has serious problems noticing the similarities between these situations, but tons of people seem not to realise there are any differences. The priority with terminology must be to separate the meanings and make it absolutely clear that these are not the same thing and need not be treated in the same way. Giving them similar names is nearly the worst thing that could be done, second only to leaving the situation as it is.
If you were to propose a systematic terminology for decision-theoretric dilemmas, that would be a different matter. I think I would disagree with you, the field is young and we don’t have a good enough picture of the space of possible problems, a systemic scheme risks reducing our ability to think beyond it.
But that is not what is being suggested, what is being suggested is creating an ad-hoc confusion generator by making deliberately similar terms for different situations.
This might all be rationalisation, but thats my best guess for why the situations feel different to me.
I agree with your analysis regarding the difference between systematic naming systems and merely similar naming. That said, the justification for more clearly separating Pascal’s mugging and this other unnamed situation does strike me as a political decision or rationalization. If the real world impact of people’s misunderstanding were beneficial for the AI friendly cause, I doubt if anyone here would be making much ado about it. I would be in favor of renaming moissanite to diamand if this would help avert our ongoing malinvestment in clear glittery rocks to the tune of billions of dollars and numerous lives, so political reasons can perhaps be justified in some situations.
I would agree that it is to some extent political. I don’t think its very dark artsy though, because it seems to be a case of getting rid of an anti-FAI misunderstanding rather than creating a pro-FAI misunderstanding.
Would the English language really be better if rubies were called diyermands?
I suspect it would be. The first time one encounters the word “ruby”, you have only context to go off of. But if the word sounded like “diamond”, then you could also make a tentative guess that the referent is also similar.
Do you really think this!? I admit to being extremely surprised to find anyone saying this.
If rubies were called diyermands it seems to me that people wouldn’t guess what it was when they heard it, they would simply guess that they had misheard ‘diamond’, especially since it would almost certainly be a context where that was plausible, most people would probably still have to have the word explained to them.
Furthermore, once we had the definition, we would be endlessly mixing them up, given that they come up in exactly the same context. Words are used many times, but only need to be learned once, so getting the former unambiguous is far more important.
The word ‘ruby’ exists primarily to distinguish them from things like diamonds, you can usually guess that they’re not cows from context. Replacing it with diyermand causes it to fail at its main purpose.
EDIT:
To give an example from my own field, in maths we have the terms ‘compact’ and ‘sequentially compact’ for types of topological space. The meanings are similar but not the same, you can find spaces satisfying one but not the other, but most ‘nice’ spaces have both or neither.
If your theory is correct, this situation is good, because it will allow people to form a plausible guess at what ‘compact’ means if they already know ‘sequentially compact’ (this is almost always they order a student meets them). Indeed, they do always form a plausible guess, and that guess is ‘the two terms mean the same thing’. This guess seems so plausible, they never question it and go off believing the wrong thing. In my case this lasted about 6 months before someone undeluded me, even when I learned the real definition of compactness, I assumed they were provably equivalent.
Had their names been totally different, I would have actually asked what it meant when I first heard it, and would never have had any misunderstandings, and several others I know would have avoided them as well. This seems unambiguously better.
Hm, that’s a good point, I’ve changed my opinion about this case.
When I wrote my comment, I was thinking primarily of words that share a common prefix or suffix, which tends to imply that they refer to things that share the same category but are not the same thing. “English” and “Spanish”, for example.
But yeah, “diyer” is too close to “die” to be easily distinguishable. Maybe “rubemond”?
But yeah, “diyer” is too close to “die” to be easily distinguishable. Maybe “rubemond”?
I could see the argument for that, provided we also had saphmonds, emmonds etc… Otherwise you run the risk of claiming a special connection that doesn’t exist.
2) Some of the arguments against Pascal’s Wager and Pascal’s Mugging don’t depend on the probability. For instance, Pascal’s Wager has the “worshipping the wrong god” problem—what if there’s a god who prefers that he not be worshipped and damns worshippers to Hell? Even if there’s a 99% chance of a god existing, this is still a legitimate objection (unless you want to say there’s a 99% chance specifically of one type of god).
That argument is isomorphic to the one discussed in the post here:
“Hmm...” she says. “I hadn’t thought of that. But what if these equations are right, and yet somehow, everything I do is exactly balanced, down to the googolth decimal point or so, with respect to how it impacts the chance of modern-day Earth participating in a chain of events that leads to creating an intergalactic civilization?”
“How would that work?” you say. “There’s only seven billion people on today’s Earth—there’s probably been only a hundred billion people who ever existed total, or will exist before we go through the intelligence explosion or whatever—so even before analyzing your exact position, it seems like your leverage on future affairs couldn’t reasonably be less than a one in ten trillion part of the future or so.”
Essentially, it’s hard to argue that the probabilities you assign should be balanced so exactly, and thus (if you’re an altruist) Pascal’s Wager exhorts you either to devote your entire existence to proselytizing for some god, or proselytizing for atheism, depending on which type of deity seems to you to have the slightest edge in probability (maybe with some weighting for the awesomeness of their heavens and awfulness of their hells).
So that’s why you still need a mathematical/epistemic/decision-theoretic reason to reject Pascal’s Wager and Mugger.
What you have is a divergent sum whose sign will depend to the order of summation, so maybe some sort of re-normalization can be applied to make it balance itself out in absence of evidence.
Actually, there is no order of summation in which the sum will converge, since the terms get arbitrary large. The theorem you are thinking of applies to conditionally convergent series, not all divergent series.
Strictly speaking, you don’t always need the sums to converge. To choose between two actions you merely need the sign of difference between utilities of two actions, which you can represent with divergent sum. The issue is that it is not clear how to order such sum or if it’s sign is even meaningful in any way.
Without discussing the merits of your proposal, this is something that clearly falls under “mathematical/epistemic/decision-theoretic reason to reject Pascal’s Wager and Mugger”, so I don’t understand why you left that comment here.
If someone suggests to me that they have the ability to save 3^^^3 lives, and I assign this a 1/3^^^3 probability, and then they open a gap in the sky at billions to one odds, I would conclude that it is still extremely unlikely that they can save 3^^^3 lives. However, it is possible that their original statement is false and yet it would be worth giving them five dollars because they would save a billion lives. Of course, this would require further assumptions on whether people are likely to do things that they have not said they would do, but are weaker versions of things they did say they would do but are not capable of.
Also, I would assign lower probabilities when they claim they could save more people, for reasons that have nothing to do with complexity. For instance, “the more powerful a being is, the less likely he would be interested in five dollars” or :”a fraudster would wish to specify a large number to increase the chance that his fraud succeeds when used on ordinary utility maximizers, so the larger the number, the greater the comparative likelihood that the person is fraudulent”.
1) Sometimes what you may actually be seeing is disagreement on whether the hypothesis has a low probability.
2) Some of the arguments against Pascal’s Wager and Pascal’s Mugging don’t depend on the probability. For instance, Pascal’s Wager has the “worshipping the wrong god” problem—what if there’s a god who prefers that he not be worshipped and damns worshippers to Hell? Even if there’s a 99% chance of a god existing, this is still a legitimate objection (unless you want to say there’s a 99% chance specifically of one type of god).
3) In some cases, it may be technically true that there is no low probability involved but there may be some other small number that the size of the benefit is multiplied by. For instance, most people discount events that happen far in the future. A highly beneficial event that happens far in the future would have the benefit multiplied by a very small number when considering discounting.
Of course in cases 2 and 3 that is not technically Pascal’s mugging by the original definition, but I would suggest the definition should be extended to include such cases. Even if not, they should at least be called something that acknowledges the similarity, like “Pascal-like muggings”.
1) It’s been applied to cryonic preservation, fer crying out loud. It’s reasonable to suspect that the probability of that working is low, but anyone who says with current evidence that the probability is beyond astronomically low is being too silly to take seriously.
The benefit of cryonic preservation isn’t astronomically high, though, so you don’t need a probability that is beyond astronomically low. First of all,even an infinitely long life after being revived only has a finite present value, and possibly a very low one, because of discounting. Second, the benefit from cryonics is the benefit you’d gain from being revived after being cryonically preserved, minus the benefit that you’d gain from being revived after not cryonically preserved. (A really advanced society might be able to simulate us. If simulations count as us, simulating us counts as reviving us without the need for cryonic preservation.)
I do not think that you have gotten Luke’s point. He was addressing your point #1, not trying to make a substantive argument in favor of cryonics.
I don’t think that either Pascal’s Wager or Pascal’s Mugging requires a probability that is astronomically low. It just requires that the size of the purported benefit be large enough that it overwhelms the low probability of the event.
No, otherwise taking good but long-shot bets would be a case of Pascal’s Mugging.
It needs to involve a breakdown in the math because you’re basically trying to evaluate infinity/infinity
Any similarities are arguments for giving them a maximally different name to avoid confusion, not a similar one. Would the English language really be better if rubies were called diyermands?
Chemistry would not be improved by providing completely different names to chlorate and perchlorate (e.g. chlorate and sneblobs). Also, I think English might be better if rubies were called diyermands. If all of the gemstones were named something that followed a scheme similar to diamonds, that might be an improvement.
I disagree. Communication can be noisy, and if a bit of noise replaces a word with a word in a totally different semantic class the error can be recovered, whereas if it replaces it with a word in the similar class it can’t. See the last paragraph in myl’s comment to this comment.
Humans have the luxury of neither perfect learning nor perfect recall. In general, I find that my ability to learn and ability to recall words are much more limiting generally speaking than noisy communication channels. I think that there are other sources of redundancy in human communication that make noise less of an issue. For example, if I’m not sure if someone said “chlorate” or “perchlorate” often the ambiguity would be obvious, such as if it is clear that they had mumbled so I wasn’t quite sure what they said. In the case of the written word, Chemistry and context provide a model for things which adds as a layer of redundancy, similar to the language model described in the post you linked to.
It would take me at least twice as long to memorize random/unique alternatives to hypochlorite, chlorite, chlorate, perchlorate, multiplied by all the other oxyanion series. It would take me many times as long to memorize unique names for every acetyl compound, although I obviously acknowledge that Chemistry is the best case scenario for my argument and worst case scenario for yours. In the case of philosophy, I still think there are advantages to learning and recall for similar things to be named similarly. Even in the case of “Pascal’s mugging” vs. “Pascal’s wager”, I believe that it is easier to recall and thus easier to have cognition about in part because of the naming connection between the two, despite the fact that these are two different things.
Note that I am not saying I am in favor of calling any particular thing “Pascal-like muggings,” which draws an explicit similarity between the two, all I’m saying is that choosing a “maximally different name to avoid confusion” strikes me as being less ideal, and that if you called it a Jiro’s mugging or something, that would more than enough semantic distance between the ideas.
Okay, thats actually a good example. This caused me to re-think my position. After thinking, I’m still not sure that the analogy is actually valid though.
In chemistry, we have a systemic naming scheme. Systematic name schemes are good, because they let us guess word meanings without having to learn them. In a difficult field which most people learn only as adults if at all, this is a very good thing. I’m no chemist, but if I had to guess the words chlorate and perchlorate to cause confusion sometimes, but that this price is overall worth paying for a systemic naming scheme.
For gemstones, we do not currently have a systematic naming scheme. I’m not entirely sure that bringing one in would be good, there aren’t all that many common gemstones that we’re likely to forget them and frankly if it ain’t broke don’t fix it, but I’m not sure it would be bad either.
What would not be good would be to simply rename rubies to diyermands without changing anything else. This would not only result in misunderstandings, but generate the false impression that rubies and diamonds have something special in common as distinct from Sapphires and Emeralds (I apologise for my ignorance if this is in fact the case).
But at least in the case of gemstones we do not already have a serious problem, I do not know of any major epistemic failures floating around to do with the diamond-ruby distinction.
In the case of Pascal’s mugging, we have a complete epistemic disaster, a very specific very useful term have been turned into a useless bloated red-giant word, laden with piles of negative connotations and no actual meaning beyond ‘offer of lots of utility that I need an excuse to ignore’.
I know of almost nobody who has serious problems noticing the similarities between these situations, but tons of people seem not to realise there are any differences. The priority with terminology must be to separate the meanings and make it absolutely clear that these are not the same thing and need not be treated in the same way. Giving them similar names is nearly the worst thing that could be done, second only to leaving the situation as it is.
If you were to propose a systematic terminology for decision-theoretric dilemmas, that would be a different matter. I think I would disagree with you, the field is young and we don’t have a good enough picture of the space of possible problems, a systemic scheme risks reducing our ability to think beyond it.
But that is not what is being suggested, what is being suggested is creating an ad-hoc confusion generator by making deliberately similar terms for different situations.
This might all be rationalisation, but thats my best guess for why the situations feel different to me.
I agree with your analysis regarding the difference between systematic naming systems and merely similar naming. That said, the justification for more clearly separating Pascal’s mugging and this other unnamed situation does strike me as a political decision or rationalization. If the real world impact of people’s misunderstanding were beneficial for the AI friendly cause, I doubt if anyone here would be making much ado about it. I would be in favor of renaming moissanite to diamand if this would help avert our ongoing malinvestment in clear glittery rocks to the tune of billions of dollars and numerous lives, so political reasons can perhaps be justified in some situations.
I would agree that it is to some extent political. I don’t think its very dark artsy though, because it seems to be a case of getting rid of an anti-FAI misunderstanding rather than creating a pro-FAI misunderstanding.
I suspect it would be. The first time one encounters the word “ruby”, you have only context to go off of. But if the word sounded like “diamond”, then you could also make a tentative guess that the referent is also similar.
Do you really think this!? I admit to being extremely surprised to find anyone saying this.
If rubies were called diyermands it seems to me that people wouldn’t guess what it was when they heard it, they would simply guess that they had misheard ‘diamond’, especially since it would almost certainly be a context where that was plausible, most people would probably still have to have the word explained to them.
Furthermore, once we had the definition, we would be endlessly mixing them up, given that they come up in exactly the same context. Words are used many times, but only need to be learned once, so getting the former unambiguous is far more important.
The word ‘ruby’ exists primarily to distinguish them from things like diamonds, you can usually guess that they’re not cows from context. Replacing it with diyermand causes it to fail at its main purpose.
EDIT:
To give an example from my own field, in maths we have the terms ‘compact’ and ‘sequentially compact’ for types of topological space. The meanings are similar but not the same, you can find spaces satisfying one but not the other, but most ‘nice’ spaces have both or neither.
If your theory is correct, this situation is good, because it will allow people to form a plausible guess at what ‘compact’ means if they already know ‘sequentially compact’ (this is almost always they order a student meets them). Indeed, they do always form a plausible guess, and that guess is ‘the two terms mean the same thing’. This guess seems so plausible, they never question it and go off believing the wrong thing. In my case this lasted about 6 months before someone undeluded me, even when I learned the real definition of compactness, I assumed they were provably equivalent.
Had their names been totally different, I would have actually asked what it meant when I first heard it, and would never have had any misunderstandings, and several others I know would have avoided them as well. This seems unambiguously better.
Hm, that’s a good point, I’ve changed my opinion about this case.
When I wrote my comment, I was thinking primarily of words that share a common prefix or suffix, which tends to imply that they refer to things that share the same category but are not the same thing. “English” and “Spanish”, for example.
But yeah, “diyer” is too close to “die” to be easily distinguishable. Maybe “rubemond”?
I could see the argument for that, provided we also had saphmonds, emmonds etc… Otherwise you run the risk of claiming a special connection that doesn’t exist.
We would also need to find a different word for almonds.
That argument is isomorphic to the one discussed in the post here:
Essentially, it’s hard to argue that the probabilities you assign should be balanced so exactly, and thus (if you’re an altruist) Pascal’s Wager exhorts you either to devote your entire existence to proselytizing for some god, or proselytizing for atheism, depending on which type of deity seems to you to have the slightest edge in probability (maybe with some weighting for the awesomeness of their heavens and awfulness of their hells).
So that’s why you still need a mathematical/epistemic/decision-theoretic reason to reject Pascal’s Wager and Mugger.
What you have is a divergent sum whose sign will depend to the order of summation, so maybe some sort of re-normalization can be applied to make it balance itself out in absence of evidence.
Actually, there is no order of summation in which the sum will converge, since the terms get arbitrary large. The theorem you are thinking of applies to conditionally convergent series, not all divergent series.
Strictly speaking, you don’t always need the sums to converge. To choose between two actions you merely need the sign of difference between utilities of two actions, which you can represent with divergent sum. The issue is that it is not clear how to order such sum or if it’s sign is even meaningful in any way.
Without discussing the merits of your proposal, this is something that clearly falls under “mathematical/epistemic/decision-theoretic reason to reject Pascal’s Wager and Mugger”, so I don’t understand why you left that comment here.