How can a non-profit organization ensure that its future incarnations pursue the values of its founders? How do parents ensure that their children will pursue parental values? How do individuals ensure that when older they will pursue younger-version values?
Of course the degree to which anyone can do those things is limited, Robin, (unless someone uses an engineered superintelligence to do them), but do you seriously think the questions you pose have any relevance to how the creator of a superintellence can ensure the superintelligence will follow the intentions of the creator?
If your reply is yes, please say whether you have read Eliezer’s Knowability of FAI.
Of course the degree to which anyone can do those things is limited, Robin, (unless someone uses an engineered superintelligence to do them), but do you seriously think the questions you pose have any relevance to how the creator of a superintellence can ensure the superintelligence will follow the intentions of the creator?
If your reply is yes, please say whether you have read Eliezer’s Knowability of FAI.