I remember my reaction when first reading this was “both AIs delegate their power, then a jointly trusted coinflip is made, then a new AI is constructed which maximizes one of the utility functions”. That seems to solve the problem in general.
I remember my reaction when first reading this was “both AIs delegate their power, then a jointly trusted coinflip is made, then a new AI is constructed which maximizes one of the utility functions”. That seems to solve the problem in general.