It is definitely necessary to assume linear utility for dollars. For example: suppose your (marginal) utility function for money is U($0) = 0, U($1000) = 1, U($1000000) = 2 (where $1000 and $1000000 are the amounts of money that could be in the two boxes, respectively). Furthermore, suppose Omega always correctly predicts two-boxers, so they always get $1000. However, Omega is very pessimistic about one-boxers, so only 0.2% of them get $1000000, and the average one-box value ends up being $2000.
It is then not correct to say that you should one-box. For you, the expected utility of two-boxing is exactly 1, but the expected utility of one-boxing is 0.2% x 2 = 0.004, and so one-boxing is a really stupid strategy even though the expected monetary gain is twice as high.
Edit: of course, there’s an obvious fix: compute the average utility received by people, according to your utility function, and optimize over that.
It is definitely necessary to assume linear utility for dollars. For example: suppose your (marginal) utility function for money is U($0) = 0, U($1000) = 1, U($1000000) = 2 (where $1000 and $1000000 are the amounts of money that could be in the two boxes, respectively). Furthermore, suppose Omega always correctly predicts two-boxers, so they always get $1000. However, Omega is very pessimistic about one-boxers, so only 0.2% of them get $1000000, and the average one-box value ends up being $2000.
It is then not correct to say that you should one-box. For you, the expected utility of two-boxing is exactly 1, but the expected utility of one-boxing is 0.2% x 2 = 0.004, and so one-boxing is a really stupid strategy even though the expected monetary gain is twice as high.
Edit: of course, there’s an obvious fix: compute the average utility received by people, according to your utility function, and optimize over that.