I’ve been thinking about this for a while, and I think I have some useful ideas that people here may wish to look at. You may want to look at theses posts I have written:
A summary of what I have written so far: Any agent which interacts with the world to achieve certain goals ( <==> follow some set of “terminal values”) will pursue a certain set of instrumental values, or subgoals. It is a non-trivial fact about the universe that we live in that these subgoals show a fairly weak dependence on the supergoals that motivated them. Steve Omohundro realized this in his paper on “The nature of self-improving artificial intelligence”.
I realized, independently, that this line of argument may well apply to a civilization pursuing a certain notion of “the good life”: the instrumental values that they pursue may turn out to be fairly independent of their terminal values. I quote:
Let U denote a utility function which represents some idea of what is intrinsically valuable, and write I(U) for the notion of instrumental value that U gives rise to. For any notion of value which grows with the number of people alive, “Progress”(progress in physics, engineering, economics, communication, etc) always becomes an instrumental value. For example, if R = the number of people alive who have red hair, then I(R) includes “Progress” as defined above. If Z = number of prayers which are said to the god Zeus, then I(Z) also includes “Progress”.
Anyone whose instrumental value includes progress should obviously also include “Knowledge” and it should include “Creativity”, because one moment of creative genius can equate to a huge amount of progress, and it is an inherent property of creativity that you cannot predict in advance where that creativity will come from. Certain personal (and even political) types of “Freedom”, and “Diversity” are therefore also included – because a group of people who all think in the same ways are less creative than a diverse group.
Even some kinds of intrinsic value which make no reference to people will include instrumental values which require people. For example if P = “the number of paperclips in the universe”, then I(P) includes “Knowledge” and “Progress”. But then it also includes “Creativity”, and “Freedom” and “Diversity”.
I’ve been thinking about this for a while, and I think I have some useful ideas that people here may wish to look at. You may want to look at theses posts I have written:
Transhumanism and the need for realist ethics
The road to Universal Ethics: Universal Instrumental Values
The error of conteporary ethics: values from nowhere?
A summary of what I have written so far: Any agent which interacts with the world to achieve certain goals ( <==> follow some set of “terminal values”) will pursue a certain set of instrumental values, or subgoals. It is a non-trivial fact about the universe that we live in that these subgoals show a fairly weak dependence on the supergoals that motivated them. Steve Omohundro realized this in his paper on “The nature of self-improving artificial intelligence”.
I realized, independently, that this line of argument may well apply to a civilization pursuing a certain notion of “the good life”: the instrumental values that they pursue may turn out to be fairly independent of their terminal values. I quote:
Let U denote a utility function which represents some idea of what is intrinsically valuable, and write I(U) for the notion of instrumental value that U gives rise to. For any notion of value which grows with the number of people alive, “Progress”(progress in physics, engineering, economics, communication, etc) always becomes an instrumental value. For example, if R = the number of people alive who have red hair, then I(R) includes “Progress” as defined above. If Z = number of prayers which are said to the god Zeus, then I(Z) also includes “Progress”.
Anyone whose instrumental value includes progress should obviously also include “Knowledge” and it should include “Creativity”, because one moment of creative genius can equate to a huge amount of progress, and it is an inherent property of creativity that you cannot predict in advance where that creativity will come from. Certain personal (and even political) types of “Freedom”, and “Diversity” are therefore also included – because a group of people who all think in the same ways are less creative than a diverse group.
Even some kinds of intrinsic value which make no reference to people will include instrumental values which require people. For example if P = “the number of paperclips in the universe”, then I(P) includes “Knowledge” and “Progress”. But then it also includes “Creativity”, and “Freedom” and “Diversity”.