So in this formulation, human values are explicitly considered to be dynamic and in a constant change as people accumulate new experiences and have their environment change. Say that the Victorians invent a steampunk version of the Internet; that’s going to cause them to have new kinds of experiences, which will cause changes in their values.
Both individuals and societies also have lots of different value conflicts that they will want to resolve; see e.g. the last three paragraphs of this comment. Resolving those conflicts and helping people find the most rewarding things will naturally change their values.
Now there is still a bit of a risk of value lock-in, in that the AI is postulated to use the society’s existing values as the rule that determines what kinds of adjustments to values are acceptable. But I think that there’s an inevitable tradeoff, in that we both want to allow for value evolution, and to make sure that we don’t end up in a future that would contain nothing of value (as judged by us current-day humans). Unless we are prepared to just let anything happen (in which case why bother with Friendly AI stuff in the first place?), we need to have our existing values guide some of the development process.
So in this formulation, human values are explicitly considered to be dynamic and in a constant change as people accumulate new experiences and have their environment change. Say that the Victorians invent a steampunk version of the Internet; that’s going to cause them to have new kinds of experiences, which will cause changes in their values.
Both individuals and societies also have lots of different value conflicts that they will want to resolve; see e.g. the last three paragraphs of this comment. Resolving those conflicts and helping people find the most rewarding things will naturally change their values.
Now there is still a bit of a risk of value lock-in, in that the AI is postulated to use the society’s existing values as the rule that determines what kinds of adjustments to values are acceptable. But I think that there’s an inevitable tradeoff, in that we both want to allow for value evolution, and to make sure that we don’t end up in a future that would contain nothing of value (as judged by us current-day humans). Unless we are prepared to just let anything happen (in which case why bother with Friendly AI stuff in the first place?), we need to have our existing values guide some of the development process.