I was not aware of Everitt, Leike & Hutter 2015, thank you for the reference! I only delved into decision theory a few weeks ago, so I haven’t read that much yet.
Nash equilibria come from the fact that your action depends on your opponent’s action, which depends on your action. When you assume that each player will greedily change their action if it improves their utility, the Nash equilibria are the fixpoints at which no player changes their action.
In single-agent decision theory problems, your (best) action depends on the situation you’re in, which depends on what someone predicted your action would be, which (effectively) depends on your action.
If there’s a deeper connection than this, I don’t know it. There’s a fundamental difference between the two cases, I think, because a Nash equilibrium involves multiple agents that don’t know each others’ decision process (problem statement: maximize the outputs of two functions independently), while single-agent decision theory involves just one agent (problem statement: maximize the output of one function).
I’d say that the connection is: Single-agent problems with predictors can be interpreted as sequential two-player games where the (perfect) predictor is a player who observes the action of the decision-maker and best-responds to it. In game-theoretic jargon, the predictor is a Stackelberg follower, and the decision-maker is the Stackelberg leader. (Related: (Kovarik, Oesterheld & Conitzer 2023))
What’s the utility function of the predictor? Is there necessarily a utility function for the predictor such that the predictor’s behavior (which is arbitrary) corresponds to maximizing its own utility? (Perhaps this is mentioned in the paper, which I’ll look at.)
EDIT: do you mean to reduce a 2-player game to a single-agent decision problem, instead of vice-versa?
Is there necessarily a utility function for the predictor such that the predictor’s behavior (which is arbitrary) corresponds to maximizing its own utility
You’re right, the predictor’s behavior might not be compatible with utility maximization against any beliefs. I guess we’re often interested in cases where we can think of the predictor as an agent. The predictor’s behavior might be irrational in the restrictive above sense,[1] but to the extent that we think of it as an agent, my guess is that we can still get away with using a game theoretic-flavored approach.
I was not aware of Everitt, Leike & Hutter 2015, thank you for the reference! I only delved into decision theory a few weeks ago, so I haven’t read that much yet.
Nash equilibria come from the fact that your action depends on your opponent’s action, which depends on your action. When you assume that each player will greedily change their action if it improves their utility, the Nash equilibria are the fixpoints at which no player changes their action.
In single-agent decision theory problems, your (best) action depends on the situation you’re in, which depends on what someone predicted your action would be, which (effectively) depends on your action.
If there’s a deeper connection than this, I don’t know it. There’s a fundamental difference between the two cases, I think, because a Nash equilibrium involves multiple agents that don’t know each others’ decision process (problem statement: maximize the outputs of two functions independently), while single-agent decision theory involves just one agent (problem statement: maximize the output of one function).
I’d say that the connection is: Single-agent problems with predictors can be interpreted as sequential two-player games where the (perfect) predictor is a player who observes the action of the decision-maker and best-responds to it. In game-theoretic jargon, the predictor is a Stackelberg follower, and the decision-maker is the Stackelberg leader. (Related: (Kovarik, Oesterheld & Conitzer 2023))
What’s the utility function of the predictor? Is there necessarily a utility function for the predictor such that the predictor’s behavior (which is arbitrary) corresponds to maximizing its own utility? (Perhaps this is mentioned in the paper, which I’ll look at.)
EDIT: do you mean to reduce a 2-player game to a single-agent decision problem, instead of vice-versa?
[Apologies for the delay]
You’re right, the predictor’s behavior might not be compatible with utility maximization against any beliefs. I guess we’re often interested in cases where we can think of the predictor as an agent. The predictor’s behavior might be irrational in the restrictive above sense,[1] but to the extent that we think of it as an agent, my guess is that we can still get away with using a game theoretic-flavored approach.
For instance, if the predictor is unaware of some crucial hypothesis, or applies mild optimization rather than expected value maximization