I am very skeptical of this proposal for multiple reasons:
1) The Ultimatum Game is difficult to resolve because agent A’s strategy depends on agent B’s strategy depends on A’s and there isn’t an obvious way to shortcut this infinite recursion. The difficulty has nothing to do with utility functions. Perfectly selfish rational agents won’t ever add fairness qua fairness—at best they’ll add something approximating this in order to prevent the other rejecting the offer/low-balling. Lastly, if we wanted to add in fairness, we could simply include it in the utility function, so that isn’t a critique of utility functions.
2) The True Prisoner’s Dilemma/Ultimatum Game shouldn’t be dismissed. Firstly, there are actually people who actually are that selfish. Secondly, we can end up in situations where we care almost zero for the other players, even if not quite zero. But more generally, you’re falling into The Direct Application Fallacy. These games aren’t important in and of themselves, but because of what we can deduce from them about more complicated games.
3) Thirdly, in the True Prisoner’s Dilemma, I don’t see Eliezer saying that co-operate is the right answer. If I’m wrong, can you point out where he says that. The right option on the True Prisoner’s Dilemma where your decision can’t affect the other player is to defect. Calling something a dilemma doesn’t mean to resolve it with your own values—you are only supposed to import your own values when the situation doesn’t specify them (see Please Don’t Fight the Hypothetical).
1) The notion of a “perfectly selfish rational agent” presupposes the concept of a utility function. So does the idea that agent A’s strategy must depend on agent B’s which must depend on agent A’s. It doesn’t need to depend, you can literally just do something. And that is what people do in real life. And it seems silly to call it “irrational” when the “rational” action is a computation that doesn’t converge.
2) I think humanity as a whole can be thought of as a single agent. Sure maybe you can have a person who is “approximately that selfish”, but if they are playing a game against human values, there is nothing symmetrical about that. Even if you have two selfish people playing against each other, it is in the context of a world infused by human values, and this context necessarily informs their interactions.
I realize that simple games are only a proxy for complicated games. I am attacking the idea of simple games as a proxy for attacking the idea of complicated games.
3) Eliezer definitely says that when your decision is “logically correlated” with your opponent’s decision then you should cooperate regardless of whether or not there is anything causal about the correlation. This is the essential idea of TDT/UDT. Although I think UDT does have some valuable insights, I think there is also an element of motivated reasoning in the form of “it would be nice if rational agents played (C,C) against each other in certain circumstances rather than (D,D), how can we argue that this is the case”.
I am very skeptical of this proposal for multiple reasons:
1) The Ultimatum Game is difficult to resolve because agent A’s strategy depends on agent B’s strategy depends on A’s and there isn’t an obvious way to shortcut this infinite recursion. The difficulty has nothing to do with utility functions. Perfectly selfish rational agents won’t ever add fairness qua fairness—at best they’ll add something approximating this in order to prevent the other rejecting the offer/low-balling. Lastly, if we wanted to add in fairness, we could simply include it in the utility function, so that isn’t a critique of utility functions.
2) The True Prisoner’s Dilemma/Ultimatum Game shouldn’t be dismissed. Firstly, there are actually people who actually are that selfish. Secondly, we can end up in situations where we care almost zero for the other players, even if not quite zero. But more generally, you’re falling into The Direct Application Fallacy. These games aren’t important in and of themselves, but because of what we can deduce from them about more complicated games.
3) Thirdly, in the True Prisoner’s Dilemma, I don’t see Eliezer saying that co-operate is the right answer. If I’m wrong, can you point out where he says that. The right option on the True Prisoner’s Dilemma where your decision can’t affect the other player is to defect. Calling something a dilemma doesn’t mean to resolve it with your own values—you are only supposed to import your own values when the situation doesn’t specify them (see Please Don’t Fight the Hypothetical).
1) The notion of a “perfectly selfish rational agent” presupposes the concept of a utility function. So does the idea that agent A’s strategy must depend on agent B’s which must depend on agent A’s. It doesn’t need to depend, you can literally just do something. And that is what people do in real life. And it seems silly to call it “irrational” when the “rational” action is a computation that doesn’t converge.
2) I think humanity as a whole can be thought of as a single agent. Sure maybe you can have a person who is “approximately that selfish”, but if they are playing a game against human values, there is nothing symmetrical about that. Even if you have two selfish people playing against each other, it is in the context of a world infused by human values, and this context necessarily informs their interactions.
I realize that simple games are only a proxy for complicated games. I am attacking the idea of simple games as a proxy for attacking the idea of complicated games.
3) Eliezer definitely says that when your decision is “logically correlated” with your opponent’s decision then you should cooperate regardless of whether or not there is anything causal about the correlation. This is the essential idea of TDT/UDT. Although I think UDT does have some valuable insights, I think there is also an element of motivated reasoning in the form of “it would be nice if rational agents played (C,C) against each other in certain circumstances rather than (D,D), how can we argue that this is the case”.