And just as with Euclid’s fifth, I believe, the resolution is not to keep trying harder to justify it but to ask: what happens when we drop it? What does the resulting decision theory look like? Is it consistent? Is it useful? Does it perhaps describe actual rational behavior better?
(That write-up still used expected utility maximization with regard to logical/mathematical uncertainty, with a parenthetical note “This specifically assumes that expected utility maximization is the right way to deal with mathematical uncertainty. Consider it a temporary placeholder until that problem is solved.” AFAIK that problem is still open today. @Scott Garrabrant I think this answers your “weird to have it be formalized in term of expected utility maximization”?)
UDT was (in part) the result of asking this. See my 2009 indexical uncertainty and the Axiom of Independence, which pointed out that indexical uncertainty does not satisfy the Axiom of Independence, and in the comments @Vladimir_Nesov pointed out that this extends to more general kinds of uncertainty. Then UDT was written up 2 months later informed by this discussion.
(That write-up still used expected utility maximization with regard to logical/mathematical uncertainty, with a parenthetical note “This specifically assumes that expected utility maximization is the right way to deal with mathematical uncertainty. Consider it a temporary placeholder until that problem is solved.” AFAIK that problem is still open today. @Scott Garrabrant I think this answers your “weird to have it be formalized in term of expected utility maximization”?)