Quick reply: yes, that would be it—the view “that there is an objective morality and that sufficiently smart minds will converge to understanding and obeying it. On this view, AIs will end up behaving ethically by default”.
I don’t subscribe to that view, nor to the belief that there are two attractors. I think there is just one attractor, or one default outcome—the one which you call the “colonizer” and I call a process of “local control maximization”. That is an AI goal structure that includes Bostrom’s four instrumental goals. It may have some final goal above the instrumental goals, like building paperclips, solving the Riemann hypothesis, etc. etc., but need not have one. Just like humanity probably does not have any superior goal beyond survival and multiplication, and by extension—also resource acquisition, efficiency, and creativity / technological advancement.
there is just one attractor, or one default outcome—the one which you call the “colonizer”
If this is true, then alignment is insoluble. But this view doesn’t seem to explain the rapid decay of colonialism after World War 2. Or the fact that mankind worries about animal rights...
I don’t know if worrying about animal rights should count if we simultaneously also do factory farming...
And as for the trends in human rights, democratic inclusion, expansion of welfare states, decline in violence, etc., they are real, but unfortunately they also correlate in time with increasing demand for skilled human labor. In a hypothetical future world where human labor wouldn’t matter anymore because of full automation by superhuman AGI, I fear that these trends could easily reverse (though we may actually become extinct before that takes place).
Quick reply: yes, that would be it—the view “that there is an objective morality and that sufficiently smart minds will converge to understanding and obeying it. On this view, AIs will end up behaving ethically by default”.
I don’t subscribe to that view, nor to the belief that there are two attractors. I think there is just one attractor, or one default outcome—the one which you call the “colonizer” and I call a process of “local control maximization”. That is an AI goal structure that includes Bostrom’s four instrumental goals. It may have some final goal above the instrumental goals, like building paperclips, solving the Riemann hypothesis, etc. etc., but need not have one. Just like humanity probably does not have any superior goal beyond survival and multiplication, and by extension—also resource acquisition, efficiency, and creativity / technological advancement.
If this is true, then alignment is insoluble. But this view doesn’t seem to explain the rapid decay of colonialism after World War 2. Or the fact that mankind worries about animal rights...
I don’t know if worrying about animal rights should count if we simultaneously also do factory farming...
And as for the trends in human rights, democratic inclusion, expansion of welfare states, decline in violence, etc., they are real, but unfortunately they also correlate in time with increasing demand for skilled human labor. In a hypothetical future world where human labor wouldn’t matter anymore because of full automation by superhuman AGI, I fear that these trends could easily reverse (though we may actually become extinct before that takes place).