Mapping human values is even more difficult than mapping human everyday concepts as e.g. Cyc did/tried. Put vagueness into exact symbolic form. And with vagueness I don’t mean ‘I don’t care’ but ‘related in a varying way’. Varying with respect to other relations (recursively) and varying with individual differences.
If we really tried to map human values symbolically we’d have to map each individuals values symbolically too and then symbolically aggregate that.
I don’t think that we can do that. An AGI could but that is too late.
What we can do is map human values vaguely. We could e.g. train large deep neuronal nets to learn and approximate these concepts from whatever evidence we feed it. And then look at the inferred structure whether it is sufficiently close to what we want. That way we do not have to do the mapping ourselves; only the checking.
This is the point I am trying to make—I think we should be starting some sort of mapping and having a nice long argument about it well before an AGI is realised so that humans can try and work out some sort of agreement before the AGI makes a very fast calculation.
Mapping human values is even more difficult than mapping human everyday concepts as e.g. Cyc did/tried. Put vagueness into exact symbolic form. And with vagueness I don’t mean ‘I don’t care’ but ‘related in a varying way’. Varying with respect to other relations (recursively) and varying with individual differences.
If we really tried to map human values symbolically we’d have to map each individuals values symbolically too and then symbolically aggregate that.
I don’t think that we can do that. An AGI could but that is too late.
What we can do is map human values vaguely. We could e.g. train large deep neuronal nets to learn and approximate these concepts from whatever evidence we feed it. And then look at the inferred structure whether it is sufficiently close to what we want. That way we do not have to do the mapping ourselves; only the checking.
This is the point I am trying to make—I think we should be starting some sort of mapping and having a nice long argument about it well before an AGI is realised so that humans can try and work out some sort of agreement before the AGI makes a very fast calculation.
But maybe we should only formalize the mapping process and let the AGI carry it out?