I have talked to some people who say that they value ethical reflection, and would prefer that humanity reflected for a very long time before colonizing the stars. In a sense I agree, but at the same time I can’t help but think that “reflection” is a vacuous feel-good word that has no shared common meaning.
Some forms of reflection are clearly good. Epistemic reflection is good if you are a consequentialist, since it can help you get what you want. I also agree that narrow forms of reflection can also be good. One example of a narrow form of reflection is philosophical reflection where we compare the details of two possible outcomes and then decide which one is better.
However, there are much broader forms of reflection which I’m less hesitant to endorse. Namely, the vague types of reflection, such as reflecting on whether we really value happiness, or whether we should really truly be worried about animal suffering.
I can perhaps sympathize with the intuition that we should really try to make sure that what we put into an AI is what we really want, rather than just what we superficially want. But fundamentally, I have skepticism that there is any canonical way of doing this type of reflection that leads to non-arbitrariness.
I have heard something along the lines of “I would want a reflective procedure that extrapolates my values as long as the procedure wasn’t deceiving me or had some ulterior motive” but I just don’t see how this type of reflection corresponds to any natural class. At some point, we will just have to put some arbitrariness into the value system, and there won’t be any “right answer” about how the extrapolation is done.
The vague reflections you are referring to are analogous to somebody saying “I should really exercise more” without ever doing it. I agree that the mere promise of reflection is useless.
But I do think that reflections about the vague topics are important and possible. Actively working through one’s experiences, reading relevant books, discussing questions with intelligent people can lead to epiphanies (and eventually life choices), that wouldn’t have occurred otherwise.
However, this is not done with a push of a button and these things don’t happen randomly—they will only emerge if you are prepared to invest a lot of time and energy.
All of this happens on a personal level. To use your example, somebody may conclude from his own life experience that living a life of purpose is more important to him than to live a life of happiness. How to formalize this process so that an AI could use a canonical way to achieve it (and infer somebody’s real values simply by observing) is beyond me. It would have to know a lot more about us than is comfortable for most of us.
Related to: Realism about rationality
I have talked to some people who say that they value ethical reflection, and would prefer that humanity reflected for a very long time before colonizing the stars. In a sense I agree, but at the same time I can’t help but think that “reflection” is a vacuous feel-good word that has no shared common meaning.
Some forms of reflection are clearly good. Epistemic reflection is good if you are a consequentialist, since it can help you get what you want. I also agree that narrow forms of reflection can also be good. One example of a narrow form of reflection is philosophical reflection where we compare the details of two possible outcomes and then decide which one is better.
However, there are much broader forms of reflection which I’m less hesitant to endorse. Namely, the vague types of reflection, such as reflecting on whether we really value happiness, or whether we should really truly be worried about animal suffering.
I can perhaps sympathize with the intuition that we should really try to make sure that what we put into an AI is what we really want, rather than just what we superficially want. But fundamentally, I have skepticism that there is any canonical way of doing this type of reflection that leads to non-arbitrariness.
I have heard something along the lines of “I would want a reflective procedure that extrapolates my values as long as the procedure wasn’t deceiving me or had some ulterior motive” but I just don’t see how this type of reflection corresponds to any natural class. At some point, we will just have to put some arbitrariness into the value system, and there won’t be any “right answer” about how the extrapolation is done.
The vague reflections you are referring to are analogous to somebody saying “I should really exercise more” without ever doing it. I agree that the mere promise of reflection is useless.
But I do think that reflections about the vague topics are important and possible. Actively working through one’s experiences, reading relevant books, discussing questions with intelligent people can lead to epiphanies (and eventually life choices), that wouldn’t have occurred otherwise.
However, this is not done with a push of a button and these things don’t happen randomly—they will only emerge if you are prepared to invest a lot of time and energy.
All of this happens on a personal level. To use your example, somebody may conclude from his own life experience that living a life of purpose is more important to him than to live a life of happiness. How to formalize this process so that an AI could use a canonical way to achieve it (and infer somebody’s real values simply by observing) is beyond me. It would have to know a lot more about us than is comfortable for most of us.