However, this strongly limits the space of possible aggregated agents. Imagine two EUMs, Alice and Bob, whose utilities are each linear in how much cake they have. Suppose they’re trying to form a new EUM whose utility function is a weighted average of their utility functions. Then they’d only have three options:
Form an EUM which would give Alice all the cakes (because it weights Alice’s utility higher than Bob’s)
Form an EUM which would give Bob all the cakes (because it weights Bob’s utility higher than Alice’s)
Form an EUM which is totally indifferent about the cake allocation between them (which would allocate cakes arbitrarily, and could be swayed by the tiniest incentive to give all Alice’s cakes to Bob, or vice versa)
There was a recent paper which defined a loss minimising the self other overlap. Broadly, the loss is defined something like the difference between activations referencing itself and activations referencing other agents.
Does this help here? If Alice and Bob had both been built using SOO losses they’d always consistently assign an equivalent amount of cake to each other. I get that this breaks the initial assumption that Alice and Bob each have linear utilities but it seems like a nice way to break it in a way that ensures the best possible result for all parties.
There was a recent paper which defined a loss minimising the self other overlap. Broadly, the loss is defined something like the difference between activations referencing itself and activations referencing other agents.
Does this help here? If Alice and Bob had both been built using SOO losses they’d always consistently assign an equivalent amount of cake to each other. I get that this breaks the initial assumption that Alice and Bob each have linear utilities but it seems like a nice way to break it in a way that ensures the best possible result for all parties.