Updating the importance of reducing the chance of a misaligned AI becoming space-faring upwards
does this effectively imply that the notion of alignment in this context needs to be non-anthropocentric and not formulated in terms of human values?
(I mean, the whole approach assumes that “alien Space-Faring Civilizations” would do fine (more or less), and it’s important not to create something hostile to them.)
The post is about implications for impartial longtermists. So either under moral realism it means something like finding the best values to pursue. And under moral anti realism it means that an impartial utility function is kind of symmetrical with aliens. For example if you value something only because humans value it, then an impartial version is to also value things that alien value only because their species value it.
Though because of reasons introduced in The Convergent Path to the Stars, I think that these implications are also relevant for non-impartial longtermists.
does this effectively imply that the notion of alignment in this context needs to be non-anthropocentric and not formulated in terms of human values?
(I mean, the whole approach assumes that “alien Space-Faring Civilizations” would do fine (more or less), and it’s important not to create something hostile to them.)
The implications are stronger in that case right.
The post is about implications for impartial longtermists. So either under moral realism it means something like finding the best values to pursue. And under moral anti realism it means that an impartial utility function is kind of symmetrical with aliens. For example if you value something only because humans value it, then an impartial version is to also value things that alien value only because their species value it.
Though because of reasons introduced in The Convergent Path to the Stars, I think that these implications are also relevant for non-impartial longtermists.