Standard theory around here is that you should reject offers where you’d get less than $5.
...
standard theory around here is also that nobody should make or give in to threats
You’re massively and incorrectly overgeneralizing the standard theory. In ideal rational-actor perfect predictor scenarios, one should be known as the type of person who doesn’t take bad deals, and the only way to do that against perfect predictors is to BE the type of person who doesn’t take bad deals.
The two things which don’t apply in your post: 1) you have no reason to believe the other person knows you or that your intent has any impact on their offer. They’re not a perfect predictor, so turning down their $2 just costs you $2, without changing the probability that they offer more in other universes or future interactions.
2) It only applies to certain types of threat (where it’s not a direct causal clear action). Nobody’s advocating you withhold your wallet from an armed mugger.
I don’t understand either of your two corrections. For #1, is it really usually a requirement that the person making you the offer be a perfect predictor? What about a human-level predictor in an iterated game? Yudkowsky writes that on dath ilan, people use The Algorithm, which is to reject unfair offers with high enough probability to make it a bad gamble for the offer-maker. Dath ilan is populated by humans, not perfect predictors, so I think it’s fair to say standard theory says that this applies to humans.
Or, well, okay, maybe I should say “the zeitgeist of Lesswrong” instead of “standard theory”. In which case, my post is challenging our zeitgeist’s understanding of these two principles, not some theorem with much stricter requirements for who it applies to.
For #2, are you sure nobody’s advocating that? I think Planecrash has a plot element that demonstrates that people do advocate this (spoilers for Planecrash, obviously):
Keltham wants to destroy the entire universe because he’d rather do that than let Hell continue to exist. He realizes that the gods may bargain him down from that by offering to just fix Hell, but ~ignores this consideration because if he fixates on that and does anything differently in pursuit of such bargaining, then he’d be making a Threat instead of just maximizing his utility function, and the gods would predictably ignore him. Carissa is around too, but he rejects several possible ways she could influence his project on the grounds that, since she doesn’t prefer to destroy the universe and only wants Hell to be fixed, then if he lets her influence him in the wrong way, the gods might consider Keltham’s universe-destruction game to ultimately be a Threat made by Carissa, and predictably ignore it.
This seems very analogous to the mugging case. If it’s different, what’s the difference? Mind you that the person making the threat is a (very smart) human, not a perfect predictor.
Alternatively, what does the zeitgiest around here say about the human-governance principle “don’t negotiate with terrorists”? That one is unambiguously applied to humans. Do we generally disagree with it for that reason?
I’m just not convinced that my post frames either of these two things incorrectly. I think people around here do generally say that human beings should both reject unfair Ultimatum Game offers and not give in to threats.
...
You’re massively and incorrectly overgeneralizing the standard theory. In ideal rational-actor perfect predictor scenarios, one should be known as the type of person who doesn’t take bad deals, and the only way to do that against perfect predictors is to BE the type of person who doesn’t take bad deals.
The two things which don’t apply in your post:
1) you have no reason to believe the other person knows you or that your intent has any impact on their offer. They’re not a perfect predictor, so turning down their $2 just costs you $2, without changing the probability that they offer more in other universes or future interactions.
2) It only applies to certain types of threat (where it’s not a direct causal clear action). Nobody’s advocating you withhold your wallet from an armed mugger.
I don’t understand either of your two corrections. For #1, is it really usually a requirement that the person making you the offer be a perfect predictor? What about a human-level predictor in an iterated game? Yudkowsky writes that on dath ilan, people use The Algorithm, which is to reject unfair offers with high enough probability to make it a bad gamble for the offer-maker. Dath ilan is populated by humans, not perfect predictors, so I think it’s fair to say standard theory says that this applies to humans.
Or, well, okay, maybe I should say “the zeitgeist of Lesswrong” instead of “standard theory”. In which case, my post is challenging our zeitgeist’s understanding of these two principles, not some theorem with much stricter requirements for who it applies to.
For #2, are you sure nobody’s advocating that? I think Planecrash has a plot element that demonstrates that people do advocate this (spoilers for Planecrash, obviously):
Keltham wants to destroy the entire universe because he’d rather do that than let Hell continue to exist. He realizes that the gods may bargain him down from that by offering to just fix Hell, but ~ignores this consideration because if he fixates on that and does anything differently in pursuit of such bargaining, then he’d be making a Threat instead of just maximizing his utility function, and the gods would predictably ignore him. Carissa is around too, but he rejects several possible ways she could influence his project on the grounds that, since she doesn’t prefer to destroy the universe and only wants Hell to be fixed, then if he lets her influence him in the wrong way, the gods might consider Keltham’s universe-destruction game to ultimately be a Threat made by Carissa, and predictably ignore it.
This seems very analogous to the mugging case. If it’s different, what’s the difference? Mind you that the person making the threat is a (very smart) human, not a perfect predictor.
Alternatively, what does the zeitgiest around here say about the human-governance principle “don’t negotiate with terrorists”? That one is unambiguously applied to humans. Do we generally disagree with it for that reason?
I’m just not convinced that my post frames either of these two things incorrectly. I think people around here do generally say that human beings should both reject unfair Ultimatum Game offers and not give in to threats.