A good first step for these agents is to share each’s values with the other
This is a strong assertion that I do not believe is justified.
If you are an agent with this view, then I can take advantage by sending you an altered version of my values such that the altered version’s Nash equilibrium (or plural) are all in my favor compared to the Nash equilibria of the original game.
(You can mitigate this to an extent by requiring that both parties precommit to their values… in which case I predict what your values will be and use this instead, committing to a version of my values altered according to said prediction. Not perfect, but still arguably better.)
(Of course, this has other issues if the other agent is also doing this...)
This is a strong assertion that I do not believe is justified.
If you are an agent with this view, then I can take advantage by sending you an altered version of my values such that the altered version’s Nash equilibrium (or plural) are all in my favor compared to the Nash equilibria of the original game.
(You can mitigate this to an extent by requiring that both parties precommit to their values… in which case I predict what your values will be and use this instead, committing to a version of my values altered according to said prediction. Not perfect, but still arguably better.)
(Of course, this has other issues if the other agent is also doing this...)