As it turns out, you’re right! Yesterday I discussed this issue with Caspar Oesterheld (one of the authors). Indeed, his answer to this objection is that they believe there probably are more positively than negatively correlated agents. Some arguments for that are evolutionary pressures and the correlation between decision theory and values you mention. In this post, I was implicitly relying on digital minds being crazy enough as for a big fraction of them to be negatively correlated to us. This could plausibly be the case in extortion/malevolent actors scenarios, but I don’t have any arguments for that being probable enough.
In fact, I had already come up with a different objection to my argument. And the concept of negatively correlated agents is generally problematic for other reasons. I’ll write another post presenting these and other considerations when I have the time (probably the end of this month). I’ll also go over Greaves [2016], thank you for that resource!
Thank you for your comment, Sylvester!
As it turns out, you’re right! Yesterday I discussed this issue with Caspar Oesterheld (one of the authors). Indeed, his answer to this objection is that they believe there probably are more positively than negatively correlated agents. Some arguments for that are evolutionary pressures and the correlation between decision theory and values you mention. In this post, I was implicitly relying on digital minds being crazy enough as for a big fraction of them to be negatively correlated to us. This could plausibly be the case in extortion/malevolent actors scenarios, but I don’t have any arguments for that being probable enough.
In fact, I had already come up with a different objection to my argument. And the concept of negatively correlated agents is generally problematic for other reasons. I’ll write another post presenting these and other considerations when I have the time (probably the end of this month). I’ll also go over Greaves [2016], thank you for that resource!
Ah, nice. I was just about to recommend sections 2.6.2 and 3 of Multiverse-wide Cooperation via Correlated Decision Making by Caspar.
Nice, thank you! I will delve into that one as well when I have the time :-)