with the exception of people who decided to gamble on being part of the elite in outcome B
Game-theoretically, there’s a better way. Assume that after winning the AI race, it is easy to figure out everyone else’s win probability, utility function and what they would do if they won. Human utility functions have diminishing returns, so there’s opportunity for acausal trade. Human ancestry gives a common notion of fairness, so the bargaining problem is easier than with aliens.
Most of us care some even about those who would take all for themselves, so instead of giving them the choice between none and a lot, we can give them the choice between some and a lot—the smaller their win prob, the smaller the gap can be while still incentivizing cooperation.
Therefore, the AI race game is not all or nothing. The more win probability lands on parties that can bargain properly, the less multiversal utility is burned.
Good point, acausal trade can at least ameliorate the problem, pushing towards atomic alignment. However, we understand acausal trade too poorly to be highly confident it will work. And, “making acausal trade work” might in itself be considered outside of the desiderata of atomic alignment (since it involves multiple AIs). Moreover, there are also actors that have a very low probability of becoming TAI users but whose support is beneficial for TAI projects (e.g. small donors). Since they have no counterfactual AI to bargain on their behalf, it is less likely acausal trade works here.
Yeah, I basically hope that enough people care about enough other people that some of the wealth ends up trickling down to everyone. Win probability is basically interchangeable with other people caring about you and your ressources across the multiverse. Good thing the cosmos is so large.
I don’t think making acausal trade work is that hard. All that is required is:
That the winner cares about the counterfactual versions of himself that didn’t win, or equivalently, is unsure whether they’re being simulated by another winner. (huh, one could actually impact this through memetic work today, though messing with people’s preferences like that doesn’t sound friendly)
That they think to simulate alternate winners before they expand too far to be simulated.
Game-theoretically, there’s a better way. Assume that after winning the AI race, it is easy to figure out everyone else’s win probability, utility function and what they would do if they won. Human utility functions have diminishing returns, so there’s opportunity for acausal trade. Human ancestry gives a common notion of fairness, so the bargaining problem is easier than with aliens.
Most of us care some even about those who would take all for themselves, so instead of giving them the choice between none and a lot, we can give them the choice between some and a lot—the smaller their win prob, the smaller the gap can be while still incentivizing cooperation.
Therefore, the AI race game is not all or nothing. The more win probability lands on parties that can bargain properly, the less multiversal utility is burned.
Good point, acausal trade can at least ameliorate the problem, pushing towards atomic alignment. However, we understand acausal trade too poorly to be highly confident it will work. And, “making acausal trade work” might in itself be considered outside of the desiderata of atomic alignment (since it involves multiple AIs). Moreover, there are also actors that have a very low probability of becoming TAI users but whose support is beneficial for TAI projects (e.g. small donors). Since they have no counterfactual AI to bargain on their behalf, it is less likely acausal trade works here.
Yeah, I basically hope that enough people care about enough other people that some of the wealth ends up trickling down to everyone. Win probability is basically interchangeable with other people caring about you and your ressources across the multiverse. Good thing the cosmos is so large.
I don’t think making acausal trade work is that hard. All that is required is:
That the winner cares about the counterfactual versions of himself that didn’t win, or equivalently, is unsure whether they’re being simulated by another winner. (huh, one could actually impact this through memetic work today, though messing with people’s preferences like that doesn’t sound friendly)
That they think to simulate alternate winners before they expand too far to be simulated.