It seems that you think there are two tiers of values, one consisting of provincial human values, and another consisting of the true universal values like “exploring the landscape of possible worlds”. You worry that CEV will catch only the first group of values.
From where I stand, this is just a mistaken question; the values you worry will be lost are provincial human values too! There’s no dividing line to miss.
I understand what you’re saying, and I’ve heard that answer before, repeatedly; and I don’t buy it.
Suppose we were arguing about the theory of evolution in the 19th century, and I said, “Look, this theory just doesn’t work, because our calculations indicate that selection doesn’t have the power necessary.” That was the state of things around the turn of the century, when genetic inheritance was assumed to be analog rather than discrete.
An acceptable answer would be to discover that genes were discrete things that an organism had just 2 copies of, and that one was often dominant, so that the equations did in fact show that selection had the necessary power.
An unacceptable answer would be to say, “What definition of evolution are you using? Evolution makes organisms evolve! If what you’re talking about doesn’t lead to more complex organisms, then it isn’t evolution.”
Just saying “Organisms become more complex over time” is not a theory of evolution. It’s more like an observation of evolution. A theory means you provide a mechanism and argue convincingly that it works. To get to a theory of CEV, you need to define what it’s supposed to accomplish, propose a mechanism, and show that the mechanism might accomplish the purpose.
You don’t have to get very far into this analysis to see why the answer you’ve given doesn’t, IMHO, work. I’ll try to post something later this afternoon on why.
I won’t get around to posting that today, but I’ll just add that I know that the intent of CEV is to solve the problems I’m complaining about. I know there are bullet points in the CEV document that say, “Renormalizing the dynamic”, “Caring about volition,” and, “Avoid hijacking the destiny of humankind.”
But I also know that the CEV document says,
Since the output of the CEV is one of the major forces shaping the future, I’m still pondering the order-of-evaluation problem to prevent this from becoming an infinite recursion.
and
It may be hard to get CEV right—come up with an AI dynamic such that our volition, as defined, is what we intuitively want. The technical challenge may be too hard; the problems I’m still working out may be impossible or ill-defined. I don’t intend to trust any design until I see that it works, and only to the extent I see that it works. Intentions are not always realized.
I think there is what you could call an order-of-execution problem, and I think there’s a problem with things being ill-defined, and I think the desired outcome is logically impossible. I could be wrong. But since Eliezer worries that this could be the case, I find it strange that Eliezer’s bulldogs are so sure that there are no such problems, and so quick to shoot down discussion of them.
This is one of the things I don’t understand: If you think everything is just a provincial human value, then why do you care? Why not play video games or watch YouTube videos instead of arguing about CEV? Is it just more fun?
(There’s a longish section trying to answer this question in the CEV document, but I can’t make sense of it.)
There’s a distinction that hasn’t been made on LW yet, between personal values and evangelical values. Western thought traditionally blurs the distinction between them, and assumes that, if you have personal values, you value other people having your values, and must go on a crusade to get everybody else to adopt your personal values.
The CEVer position is, as far as I can tell, that they follow their values because that’s what they are programmed to do. It’s a weird sort of double-think that can only arise when you act on the supposition that you have no free will with which to act. They’re talking themselves into being evangelists for values that they don’t really believe in. It’s like taking the ability to follow a moral code that you know has no outside justification from Nietzsche’s “master morality”, and combining it with the prohibition against value-creation from his “slave morality”.
There’s a distinction that hasn’t been made on LW yet, between personal values and evangelical values. Western thought traditionally blurs the distinction between them, and assumes that, if you have personal values, you value other people having your values, and must go on a crusade to get everybody else to adopt your personal values.
That’s how most values work. In general, I value human life. If someone does not share this value, and they decide to commit murder, then I would stop them if possible. If someone does not share this value, but is merely apathetic about murder rather than a potential murderer themselves, then I would cause them to share this value if possible, so there will be more people to help me stop actual murderers. So yes, at least in this case, I would act to get other people to adopt my values, or inhibit them from acting on their own values. Is this overly evangelical? What is bad about it?
In any case, history seems to indicate that “evangelizing your values” is a “universal human value”.
Groups that didn’t/don’t value evangelizing their values:
The Romans. They don’t care what you think; they just want you to pay your taxes.
The Jews. Because God didn’t choose you.
Nietzschians. Those are their values, dammit! Create your own!
Goths. (Angst-goths, not Visi-goths.) Because if everyone were a goth, they’d be just like everyone else.
We get into one sort of confusion by using particular values as examples. You talk about valuing human life. How about valuing the taste of avocados? Do you want to evangelize that? That’s kind of evangelism-neutral. How about the preferences you have that make one particular private place, or one particular person, or other limited resource, special to you? You don’t want to evangelize those preferences, or you’d have more competition. Is the first sort of value the only one CEV works with? How does it make that distinction?
We get into another sort of confusion by not distinguishing between the values we hold as individuals, the values we encourage our society to hold, and the values we want God to hold. The kind of values you want your God to hold are very different from the kind of values you want people to hold, in the same way that you want the referee to have different desires than the players. CEV mushes these two very different things together.
Good points. I haven’t thoroughly read the CEV document yet, so I don’t know if there is any discussion of this, but it does seem that it should make a distinction between those different types of values and preferences.
Or to take one step back:
It seems that you think there are two tiers of values, one consisting of provincial human values, and another consisting of the true universal values like “exploring the landscape of possible worlds”. You worry that CEV will catch only the first group of values.
From where I stand, this is just a mistaken question; the values you worry will be lost are provincial human values too! There’s no dividing line to miss.
I understand what you’re saying, and I’ve heard that answer before, repeatedly; and I don’t buy it.
Suppose we were arguing about the theory of evolution in the 19th century, and I said, “Look, this theory just doesn’t work, because our calculations indicate that selection doesn’t have the power necessary.” That was the state of things around the turn of the century, when genetic inheritance was assumed to be analog rather than discrete.
An acceptable answer would be to discover that genes were discrete things that an organism had just 2 copies of, and that one was often dominant, so that the equations did in fact show that selection had the necessary power.
An unacceptable answer would be to say, “What definition of evolution are you using? Evolution makes organisms evolve! If what you’re talking about doesn’t lead to more complex organisms, then it isn’t evolution.”
Just saying “Organisms become more complex over time” is not a theory of evolution. It’s more like an observation of evolution. A theory means you provide a mechanism and argue convincingly that it works. To get to a theory of CEV, you need to define what it’s supposed to accomplish, propose a mechanism, and show that the mechanism might accomplish the purpose.
You don’t have to get very far into this analysis to see why the answer you’ve given doesn’t, IMHO, work. I’ll try to post something later this afternoon on why.
I won’t get around to posting that today, but I’ll just add that I know that the intent of CEV is to solve the problems I’m complaining about. I know there are bullet points in the CEV document that say, “Renormalizing the dynamic”, “Caring about volition,” and, “Avoid hijacking the destiny of humankind.”
But I also know that the CEV document says,
and
I think there is what you could call an order-of-execution problem, and I think there’s a problem with things being ill-defined, and I think the desired outcome is logically impossible. I could be wrong. But since Eliezer worries that this could be the case, I find it strange that Eliezer’s bulldogs are so sure that there are no such problems, and so quick to shoot down discussion of them.
This is one of the things I don’t understand: If you think everything is just a provincial human value, then why do you care? Why not play video games or watch YouTube videos instead of arguing about CEV? Is it just more fun?
(There’s a longish section trying to answer this question in the CEV document, but I can’t make sense of it.)
There’s a distinction that hasn’t been made on LW yet, between personal values and evangelical values. Western thought traditionally blurs the distinction between them, and assumes that, if you have personal values, you value other people having your values, and must go on a crusade to get everybody else to adopt your personal values.
The CEVer position is, as far as I can tell, that they follow their values because that’s what they are programmed to do. It’s a weird sort of double-think that can only arise when you act on the supposition that you have no free will with which to act. They’re talking themselves into being evangelists for values that they don’t really believe in. It’s like taking the ability to follow a moral code that you know has no outside justification from Nietzsche’s “master morality”, and combining it with the prohibition against value-creation from his “slave morality”.
That’s how most values work. In general, I value human life. If someone does not share this value, and they decide to commit murder, then I would stop them if possible. If someone does not share this value, but is merely apathetic about murder rather than a potential murderer themselves, then I would cause them to share this value if possible, so there will be more people to help me stop actual murderers. So yes, at least in this case, I would act to get other people to adopt my values, or inhibit them from acting on their own values. Is this overly evangelical? What is bad about it?
In any case, history seems to indicate that “evangelizing your values” is a “universal human value”.
Groups that didn’t/don’t value evangelizing their values:
The Romans. They don’t care what you think; they just want you to pay your taxes.
The Jews. Because God didn’t choose you.
Nietzschians. Those are their values, dammit! Create your own!
Goths. (Angst-goths, not Visi-goths.) Because if everyone were a goth, they’d be just like everyone else.
We get into one sort of confusion by using particular values as examples. You talk about valuing human life. How about valuing the taste of avocados? Do you want to evangelize that? That’s kind of evangelism-neutral. How about the preferences you have that make one particular private place, or one particular person, or other limited resource, special to you? You don’t want to evangelize those preferences, or you’d have more competition. Is the first sort of value the only one CEV works with? How does it make that distinction?
We get into another sort of confusion by not distinguishing between the values we hold as individuals, the values we encourage our society to hold, and the values we want God to hold. The kind of values you want your God to hold are very different from the kind of values you want people to hold, in the same way that you want the referee to have different desires than the players. CEV mushes these two very different things together.
Good points. I haven’t thoroughly read the CEV document yet, so I don’t know if there is any discussion of this, but it does seem that it should make a distinction between those different types of values and preferences.