I’d suggest that such an agent is just extremely risk-averse. On the other hand, there are agents that are extremely risk-loving and those people “feel crazy” to me and some proportion of them haven’t really thought through the risks, but others just have different values.
I’m not clear what risk aversion has to do with it. I believe (but do not have a mathematical proof) that an agent that simply shuts up and multiplies (i.e. is risk-neutral), and properly accounts for the game theory, refuses to pay.
Shutting up and multiplying causes naive decision theorists to pay the mugger, just as naive decision-theoretic hitchhikers get left in the desert by drivers who can see that they won’t repay their help, and Omega can offer enormous amounts to naive decision theorists in Newcomb’s Problem and never have to pay.
Choosing a strategy, if done properly, refuses to pay the mugger, as it refuses all other attempts at blackmail. Come to think of it, as Eliezer has argued somewhere (perhaps in the context of Roko’s basilisk) that the correct way to handle blackmail is to have the invariant strategy of not paying, and Pascal’s Mugging is an example of blackmail, the PM conundrum he posed in 2007 should be easily solved by his current self.
Folk wisdom knows this. “Never play poker with strangers.” “Never take a strange bet from a stranger.” Damon Runyon gave a more colourful version.
I’d suggest that such an agent is just extremely risk-averse. On the other hand, there are agents that are extremely risk-loving and those people “feel crazy” to me and some proportion of them haven’t really thought through the risks, but others just have different values.
I’m not clear what risk aversion has to do with it. I believe (but do not have a mathematical proof) that an agent that simply shuts up and multiplies (i.e. is risk-neutral), and properly accounts for the game theory, refuses to pay.
Isn’t the whole issue that shutting up and multiplying causes people to pay the mugger?
Shutting up and multiplying causes naive decision theorists to pay the mugger, just as naive decision-theoretic hitchhikers get left in the desert by drivers who can see that they won’t repay their help, and Omega can offer enormous amounts to naive decision theorists in Newcomb’s Problem and never have to pay.
Choosing a strategy, if done properly, refuses to pay the mugger, as it refuses all other attempts at blackmail. Come to think of it, as Eliezer has argued somewhere (perhaps in the context of Roko’s basilisk) that the correct way to handle blackmail is to have the invariant strategy of not paying, and Pascal’s Mugging is an example of blackmail, the PM conundrum he posed in 2007 should be easily solved by his current self.
Folk wisdom knows this. “Never play poker with strangers.” “Never take a strange bet from a stranger.” Damon Runyon gave a more colourful version.