In this case, the question is what do the f_i(S_j) mean. These are expected utilities of a possible strategy, but how do you compute them? CDT, TDT and UDT would have it differently.
[...]
(Alternatively, you could give a way of estimating utility as a black box, and then consider various ways of constructing an estimate of expected utility out of it.)
The post calls the functions f_i “utility functions”, not “expected utility functions”. So, I take Giles to be pursuing your “alternative” approach. However, I don’t think that f_i(S_j) denotes the total utility of a state of the universe. It is just one of the terms used to compute such a total utility.
From the comments about additivity, I take f_i(S_j) to be the amount by which the utility of a universe to species i would increase if a planet following strategy j were added to it (while the strategies of all other planets remained unchanged), regardless of how or by whom that planet is added. Giles’s question, as I understand it, is, how should these “utility” terms be incorporated into an expected utility calculation? For example, what should the probability weights say is the probability that species i will produce a planet following the compromise strategy, given that we do?
The post calls the functions f_i “utility functions”, not “expected utility functions”.
(As an aside, some terminological confusion can result from there being a “utility relation” that compares lotteries, that can be represented by a “utility function” that takes lotteries as inputs, and separately expected utility representation of utility relation (or of “utility function”) that breaks it down into a probability distribution and a “utility function” in a different sense, that takes pure outcomes as inputs.)
However, I don’t think that f_i(S_j) denotes the total utility of a state of the universe. It is just one of the terms used to compute such a total utility.
Right.
From the comments about additivity, I take f_i(S_j) to be the amount by which the utility of a universe to species i would increase if a planet following strategy j were added to it (while the strategies of all other planets remained unchanged), regardless of how or by whom that planet is added.
Or, more usefully (since we can’t actually add planets), the utility function of aliens #k that takes a collection S of strategies for each of the planets under consideration (i.e. a state of the world) is
F_k (S) = sum_p f_k(S_p)
Then, the decision problem is to maximize expected value of F_0(S) by controlling S_0, a standard game theory setting. It’s underdetermined only to the extent PD is underdetermined, in that you should still defect against CooperationBots or DefectBots, etc.
[...]
The post calls the functions f_i “utility functions”, not “expected utility functions”. So, I take Giles to be pursuing your “alternative” approach. However, I don’t think that f_i(S_j) denotes the total utility of a state of the universe. It is just one of the terms used to compute such a total utility.
From the comments about additivity, I take f_i(S_j) to be the amount by which the utility of a universe to species i would increase if a planet following strategy j were added to it (while the strategies of all other planets remained unchanged), regardless of how or by whom that planet is added. Giles’s question, as I understand it, is, how should these “utility” terms be incorporated into an expected utility calculation? For example, what should the probability weights say is the probability that species i will produce a planet following the compromise strategy, given that we do?
You are right, I retract my comment.
(As an aside, some terminological confusion can result from there being a “utility relation” that compares lotteries, that can be represented by a “utility function” that takes lotteries as inputs, and separately expected utility representation of utility relation (or of “utility function”) that breaks it down into a probability distribution and a “utility function” in a different sense, that takes pure outcomes as inputs.)
Right.
Or, more usefully (since we can’t actually add planets), the utility function of aliens #k that takes a collection S of strategies for each of the planets under consideration (i.e. a state of the world) is
F_k (S) = sum_p f_k(S_p)
Then, the decision problem is to maximize expected value of F_0(S) by controlling S_0, a standard game theory setting. It’s underdetermined only to the extent PD is underdetermined, in that you should still defect against CooperationBots or DefectBots, etc.