I don’t know exactly how to explain this, so I will just say some stuff.
Good attitude!
I think I get your general problem with the sorts of dilemas/games you mentioned, but I didn’t quite get how it was supposed to point to problems with the idea of utility functions. I will also say some stuff.
I agree that calling something the “Blanks dilemma” is a bit more suggestive than what I’d like. Maybe “The prisoner’s game”?
Another thing. There are a handful of games where an agent who “values fairness” does better than your typical CDT agent. FDT and UDT seem to sometimes produce the result of, “Here is how you can get to a higher payout equilibrium, not by terminally valuing fairness, but my being smarter and thinking more broadly.”
Also, specifically on EY and True Prisoner’s Dilemma, I believe he’s making the particular claim, “If one was running FDT, and they knew the other person was as well, they would both choose to cooperate.”
I don’t understand what it means to say that an agent who “values fairness” does better than another agent. If two agents have different value systems, how can you say that one does better than another? Regarding EY and the Prisoner’s Dilemma, I agree that EY is making that claim but I think he is also making the claim “and this is evidence that rational agents should use FDT”.
To your first point:
If two agents had identical utility functions, except for one or two small tweaks, it feels reasonable to ask “Which of these agents got more utility/actualized it’s values more?” This might be hard to actually formalize. I’m mostly running on the intuition that sometimes humans that are pretty similar might look at another and say, “It seems like this other person is getting more of what they want than I am.”
Fair enough. Though in this case the valuing fairness is a big enough change that it makes a difference to how the agents act, so it’s not clear that it can be glossed over so easily.
Good attitude!
I think I get your general problem with the sorts of dilemas/games you mentioned, but I didn’t quite get how it was supposed to point to problems with the idea of utility functions. I will also say some stuff.
I agree that calling something the “Blanks dilemma” is a bit more suggestive than what I’d like. Maybe “The prisoner’s game”?
Another thing. There are a handful of games where an agent who “values fairness” does better than your typical CDT agent. FDT and UDT seem to sometimes produce the result of, “Here is how you can get to a higher payout equilibrium, not by terminally valuing fairness, but my being smarter and thinking more broadly.”
Also, specifically on EY and True Prisoner’s Dilemma, I believe he’s making the particular claim, “If one was running FDT, and they knew the other person was as well, they would both choose to cooperate.”
I don’t understand what it means to say that an agent who “values fairness” does better than another agent. If two agents have different value systems, how can you say that one does better than another? Regarding EY and the Prisoner’s Dilemma, I agree that EY is making that claim but I think he is also making the claim “and this is evidence that rational agents should use FDT”.
To your first point: If two agents had identical utility functions, except for one or two small tweaks, it feels reasonable to ask “Which of these agents got more utility/actualized it’s values more?” This might be hard to actually formalize. I’m mostly running on the intuition that sometimes humans that are pretty similar might look at another and say, “It seems like this other person is getting more of what they want than I am.”
Fair enough. Though in this case the valuing fairness is a big enough change that it makes a difference to how the agents act, so it’s not clear that it can be glossed over so easily.