Without commenting on whether this presentation matches the original metaethics sequence (with which I disagree), this summary argument seems both unsupported and unfalsifiable.
No evidence is given for the central claim, that humans can and are converging towards a true morality we would all agree about if only we understood more true facts.
We’re told that people in the past disagreed with us about some moral questions, but we know more and so we changed our minds and we are right while they were wrong. But no direct evidence is given for us being more right. The only way to judge who’s right in a disagreement seems to be “the one who knows more relevant facts is more right” or “the one who more honestly and deeply considered the question”. This does not appear to be an objectively measurable criterion (to say the least).
The claim that ancients, like Roman soldiers, thought slavery was morally fine because they didn’t understand how much slaves suffer is frankly preposterous. Roman soldiers (and poor Roman citizens in general) were often enslaved, and some of them were later freed (or escaped from foreign captivity). Many Romans were freedmen or their descendants—some estimate that by the late Empire, almost all Roman citizens had at least some slave ancestors. And yet somehow these people, who both knew what slavery was like and were often in personal danger of it, did not think it immoral, while white Americans in no danger of enslavement campaigned for abolition.
I’m getting really sick of this claim that Eliezer says all humans would agree on some morality under extrapolation. That claim is how we get garbage like this. At no point do I recall Eliezer saying psychopaths would definitely become moral under extrapolation. He did speculate about them possibly accepting modification. But the paper linked here repeatedly talks about ways to deal with disagreements which persist under extrapolation:
In poetic terms, our coherent extrapolated volition is our wish if we knew more, thought faster, were more the people we wished we were, had grown up farther together; where the extrapolation converges rather than diverges, where our wishes cohere rather than interfere; extrapolated as we wish that extrapolated, interpreted as we wish that interpreted. (emphasis added)
Coherence is not a simple question of a majority vote. Coherence will reflect the balance, concentration, and strength of individual volitions. A minor, muddled preference of 60% of humanity might be countered by a strong, unmuddled preference of 10% of humanity. The variables are quantitative, not qualitative.
(Naturally, Eugine Nier as “seer” downvoted all of my comments.)
The metaethics sequence does say IMNSHO that most humans’ extrapolated volitions (maybe 95%) would converge on a cluster of goals which include moral ones. It furthermore suggests that this would apply to the Romans if we chose the ‘right’ method of extrapolation, though here my understanding gets hazier. In any case, the preferences that we would loosely call ‘moral’ today, and that also survive some workable extrapolation, are what I seem to mean by “morality”.
One point about the ancient world: the Bhagavad Gita, produced by a warrior culture though seemingly not by the warrior caste, tells a story of the hero Arjuna refusing to fight until his friend Krishna convinces him. Arjuna doesn’t change his mind simply because of arguments about duty. In the climax, Krishna assumes his true form as a god of death with infinitely many heads and jaws, saying, ‘I will eat all of these people regardless of what you do. The only deed you can truly accomplish is to follow your warrior duty or dharma.’ This view seems plainly environment-dependent.
I’ve simplified it a bit for the sake of brevity and comprehension of the central idea, but yeah, it’s probably right to say that humans are all born with ABOUT the same morality equation. And also true that psychopaths’ equation is further away than most’s.
I don’t think it being unfalsifiable is a problem. I think this is more of a definition than a derivation. Morality is a fuzzy concept that we have intuitions about, and we like to formalize these sorts of things into definitions. This can’t be disproven any more than the definition of a triangle can be disproven.
What needs to be done instead is show the definition to be incoherent or that it doesn’t match our intuition.
Do you think the idea is sufficiently coherent and non-self-contradictory that the way to find out if it’s right or wrong is to look for evidence?
If it was incoherent or contradicted itself, it wouldn’t even need evidence to be disproven; we would already know it’s wrong. Have I avoided being wrong in that way?
(by the way, understanding slavery might be necessary, but not sufficient to get someone to be against it. They might also need to figure out that people are equal, too. Good point, I might need to add that note into the post).
Do you think the idea is sufficiently coherent and non-self-contradictory that the way to find out if it’s right or wrong is to look for evidence?
You do understand that debates about objective vs relative morality has been going on for millenia?
They might also need to figure out that people are equal, too
No, they don’t if they themselves are in danger of becoming slaves. Notably, a major source of slaves in the Ancient world was defeated armies. Slaves weren’t clearly different people (like the blacks were in America), anyone could become a slave if his luck turned out to be really bad.
Right. Someone could be against slavery for THEM personally without being against slavery in general if they didn’t realize that what was wrong for them was also wrong for others. That’s all I’m getting at, there.
Or do you mean that they should have opposed slavery for everybody as a sort of game theory move to reduce their chance of ever becoming a slave?
“You do understand that debates about objective vs relative morality has been going on for millenia?”
What I’m getting at here is that most moral theories are so bad you don’t even need to talk about evidence. You can show them to be wrong just because they’re incoherent or self-contradictory.
It’s a pretty low standard, but I’m asking if this theory is at least coherent and consistent enough that you have to look at evidence to know if it’s wrong, instead of just pointing at its self-defeating nature to show it’s wrong. If so, yay, it might be the best I’ve ever seen. :)
Someone could be against slavery for THEM personally without being against slavery in general if they didn’t realize that what was wrong for them was also wrong for others.
Huh? I’m against going to jail personally without being against the idea of jail in general. In any case, wasn’t your original argument that ancient Greeks and Romans just didn’t understand what does it mean to be a slave? That clearly does not hold.
most moral theories are so bad you don’t even need to talk about evidence. You can show them to be wrong just because they’re incoherent or self-contradictory.
Do you mean descriptive or prescriptive moral theories? If descriptive, humans are incoherent and self-contradictory.
Which moral theories do you have in mind? A few examples will help.
Mmm, that’s not quite the right abstraction. You’re probably against innocents going to jail in general, no?
Whereas some Roman might not care, as long as it’s no one they care about.
All I’m getting at is that the Romans didn’t think certain things were wrong, but if they were shown in a sufficiently deep way everything we know, they would be moved by it, whereas if we were shown everything they know, we would not find it persuasive of their position. Neither would they, after they had seen what we’ve seen.
I’m talking metaethics, what makes something moral, what it means for something to be moral. Failed ones include divine command theory, the “whatever contributes to human flourishing” idea, whatever makes people happy, whatever matches some platonic ideals out there somehow, whatever leads to selfish interest, etc.
if they were shown in a sufficiently deep way everything we know, they would be moved by it
That doesn’t seem obvious to me at all.
Let’s try it on gay marriage. Romans certainly knew and practiced homosexuality, same for marriage. What knowledge exactly do you want to convey to them to persuade them that gay marriage is a good thing?
I’m talking metaethics, what makes something moral
So, prescriptive. I am not sure in which way do you consider the theories “failed”—in the sense that they have not risen to the status of physics meaning being able to empirically prove all their claims? That doesn’t look to be a viable criterion. In the sense of not having taken over the world? I don’t know, the divine command theory is (or, at least, has been) pretty good at that. You probably wouldn’t want a single theory to take over the world, anyway.
What knowledge exactly do you want to convey to them to persuade them that gay marriage is a good thing?
Kind of a weird example, but I’ll assume we’re talking about the Praetorian Guard. The Romans seem to have had very little respect for women and for being penetrated. So right off the bat, having them read a lot of women’s minds might change their views. (I’m not sure if I want to classify that as knowledge, though.) They likely also have false beliefs not only about women but about the gods and stable societies. None of this seems like a cure-all, but it does seem extremely promising.
I think hairyfigment is of the belief that the Romans (and in the most coherent version of his claim you would have to say male and female) were under misconceptions about the nature of male and female minds, and believes that “a sufficiently deep way” would mean correcting all these misconceptions.
My view is that we really can’t say that as things stand. We’d have to know a lot more about the Roman beliefs about the male and female minds, and compare them against what we know to be accurate about male and female minds.
I was trying to say with my second paragraph that we specifically cannot be sure about that. My first paragraph was simply my best effort at interpreting what I think hairyfigment thinks, not a statement of what I believe to be true.
From my vague recollections I think the idea is worth looking up one way or the other. After all, a massive portion of modern culture is under the impression there are no gender differences and there are other instances of clear major misconceptions I actually can attest to throughout history. But I don’t have any idea with the Romans.
After all, a massive portion of modern culture is under the impression there are no gender differences
That’s the stupid portion of modern culture, and I’m not sure they actually, um, practice that belief. Here’s a quick suggestion: make competitive sports sex-blind :-/
Do you think the idea is sufficiently coherent and non-self-contradictory that the way to find out if it’s right or wrong is to look for evidence?
Yes, I think it is coherent.
Ideological Turing test: I think your theory is this: there is some set of values, which we shall call Morals. All humans have somewhat different sets of lower-case morals. When people make moral mistakes, they can be corrected by learning or internalizing some relevant truths (which may of course be different in each case). These truths can convince even actual humans to change their moral values for the better (as opposed to values changing only over generations), as long as these humans honestly and thoroughly consider and internalize the truths. Over historical time, humans have approached closer to true Morals, and we can hope to come yet closer, because we generally collect more and more truths over time.
the way to find out if it’s right or wrong is to look for evidence?
If you mean you don’t have any evidence for your theory yet, then how or why did you come by this theory? What facts are you trying to explain or predict with it?
Remember that by default, theories with no evidence for them (and no unexplained facts we’re looking for a theory about) shouldn’t even rise to the level of conscious consideration. It’s far, far more likely that if a theory like that comes to mind, it’s for due to motivated reasoning. For example, wanting to claim your morality is better by some objective measure than that of other people, like slavers.
by the way, understanding slavery might be necessary, but not sufficient to get someone to be against it. They might also need to figure out that people are equal, too.
That’s begging the question. Believing that “people are equal” is precisely the moral belief that you hold and ancient Romans didn’t. Not holding slaves is merely one of many results of having that belief; it’s not a separate moral belief.
But why should Romans come to believe that people are equal? What sort of factual knowledge could lead someone to such a belief, despite the usually accepted idea that should cannot be derived from is?
This is an explanation of Yudkowsky’s idea from the metaethics sequence. I’m just trying to make it accessible in language and length with lots of concept handles and examples.
Technically, you could believe that people are equally allowed to be enslaved. All people equal + it’s wrong to make me a slave = it’s wrong to make anyone a slave.
“All men are created equal” emerges from two or more basic principles people are born with. You might say: “Look, you have value, yah? And your loved ones? Would they stop having value if you forgot about them? No? They have value whether or not you know them? How did you conclude they have value? Could that have happened with other people, too? Would you then think they had value? Would they stop having value if you didn’t know them? No? Well, you don’t know them; do they have value?
You take “people I care about have value” (born with it) and combine it with “be consistent” (also born with), and you get “everyone has value.”
That’s the idea in principle, anyway. You take some things people are all born with, and they combine to make the moral insights people can figure out and teach each other, just like we do with math.
Technically, you could believe that people are equally allowed to be enslaved.
In a sense, the ancient Romans did believe this. Anyone who ended up in the same situation—either taken as a war captive or unable to pay their debts—was liable to be sold as a slave. So what makes you think your position is objectively better than theirs?
“All men are created equal” emerges from two or more basic principles people are born with. You might say: “Look, you have value, yah? And your loved ones? Would they stop having value if you forgot about them? No? They have value whether or not you know them? How did you conclude they have value? Could that have happened with other people, too? Would you then think they had value? Would they stop having value if you didn’t know them? No? Well, you don’t know them; do they have value?
This assumes without argument that “value” is something people intrinsically have or can have. If instead you view value as value-to-someone, i.e. I value my loved ones, but someone else might not value them, then there is no problem.
And it turns out that yes, most people did not have an intuition that anyone has intrinsic value just by virtue of being human. Most people throughout history assigned value only to ingroup members, to the rich and powerful, and to personally valued individuals. The idea that people are intrinsically valuable is historically very new, still in the minority today globally, and for both these reasons doesn’t seem like an idea everyone should naturally arrive at if they only try to universalize their intuitions a bit.
Technically, you could believe that people are equally allowed to be enslaved. All people equal + it’s wrong to make me a slave = it’s wrong to make anyone a slave.
Without commenting on whether this presentation matches the original metaethics sequence (with which I disagree), this summary argument seems both unsupported and unfalsifiable.
No evidence is given for the central claim, that humans can and are converging towards a true morality we would all agree about if only we understood more true facts.
We’re told that people in the past disagreed with us about some moral questions, but we know more and so we changed our minds and we are right while they were wrong. But no direct evidence is given for us being more right. The only way to judge who’s right in a disagreement seems to be “the one who knows more relevant facts is more right” or “the one who more honestly and deeply considered the question”. This does not appear to be an objectively measurable criterion (to say the least).
The claim that ancients, like Roman soldiers, thought slavery was morally fine because they didn’t understand how much slaves suffer is frankly preposterous. Roman soldiers (and poor Roman citizens in general) were often enslaved, and some of them were later freed (or escaped from foreign captivity). Many Romans were freedmen or their descendants—some estimate that by the late Empire, almost all Roman citizens had at least some slave ancestors. And yet somehow these people, who both knew what slavery was like and were often in personal danger of it, did not think it immoral, while white Americans in no danger of enslavement campaigned for abolition.
I’m getting really sick of this claim that Eliezer says all humans would agree on some morality under extrapolation. That claim is how we get garbage like this. At no point do I recall Eliezer saying psychopaths would definitely become moral under extrapolation. He did speculate about them possibly accepting modification. But the paper linked here repeatedly talks about ways to deal with disagreements which persist under extrapolation:
(Naturally, Eugine Nier as “seer” downvoted all of my comments.)
The metaethics sequence does say IMNSHO that most humans’ extrapolated volitions (maybe 95%) would converge on a cluster of goals which include moral ones. It furthermore suggests that this would apply to the Romans if we chose the ‘right’ method of extrapolation, though here my understanding gets hazier. In any case, the preferences that we would loosely call ‘moral’ today, and that also survive some workable extrapolation, are what I seem to mean by “morality”.
One point about the ancient world: the Bhagavad Gita, produced by a warrior culture though seemingly not by the warrior caste, tells a story of the hero Arjuna refusing to fight until his friend Krishna convinces him. Arjuna doesn’t change his mind simply because of arguments about duty. In the climax, Krishna assumes his true form as a god of death with infinitely many heads and jaws, saying, ‘I will eat all of these people regardless of what you do. The only deed you can truly accomplish is to follow your warrior duty or dharma.’ This view seems plainly environment-dependent.
No, you’re totally right.
I’ve simplified it a bit for the sake of brevity and comprehension of the central idea, but yeah, it’s probably right to say that humans are all born with ABOUT the same morality equation. And also true that psychopaths’ equation is further away than most’s.
I don’t think it being unfalsifiable is a problem. I think this is more of a definition than a derivation. Morality is a fuzzy concept that we have intuitions about, and we like to formalize these sorts of things into definitions. This can’t be disproven any more than the definition of a triangle can be disproven.
What needs to be done instead is show the definition to be incoherent or that it doesn’t match our intuition.
You’re right; I’ve provided no evidence.
Do you think the idea is sufficiently coherent and non-self-contradictory that the way to find out if it’s right or wrong is to look for evidence?
If it was incoherent or contradicted itself, it wouldn’t even need evidence to be disproven; we would already know it’s wrong. Have I avoided being wrong in that way?
(by the way, understanding slavery might be necessary, but not sufficient to get someone to be against it. They might also need to figure out that people are equal, too. Good point, I might need to add that note into the post).
You do understand that debates about objective vs relative morality has been going on for millenia?
No, they don’t if they themselves are in danger of becoming slaves. Notably, a major source of slaves in the Ancient world was defeated armies. Slaves weren’t clearly different people (like the blacks were in America), anyone could become a slave if his luck turned out to be really bad.
Right. Someone could be against slavery for THEM personally without being against slavery in general if they didn’t realize that what was wrong for them was also wrong for others. That’s all I’m getting at, there.
Or do you mean that they should have opposed slavery for everybody as a sort of game theory move to reduce their chance of ever becoming a slave?
“You do understand that debates about objective vs relative morality has been going on for millenia?”
What I’m getting at here is that most moral theories are so bad you don’t even need to talk about evidence. You can show them to be wrong just because they’re incoherent or self-contradictory.
It’s a pretty low standard, but I’m asking if this theory is at least coherent and consistent enough that you have to look at evidence to know if it’s wrong, instead of just pointing at its self-defeating nature to show it’s wrong. If so, yay, it might be the best I’ve ever seen. :)
Huh? I’m against going to jail personally without being against the idea of jail in general. In any case, wasn’t your original argument that ancient Greeks and Romans just didn’t understand what does it mean to be a slave? That clearly does not hold.
Do you mean descriptive or prescriptive moral theories? If descriptive, humans are incoherent and self-contradictory.
Which moral theories do you have in mind? A few examples will help.
Mmm, that’s not quite the right abstraction. You’re probably against innocents going to jail in general, no?
Whereas some Roman might not care, as long as it’s no one they care about.
All I’m getting at is that the Romans didn’t think certain things were wrong, but if they were shown in a sufficiently deep way everything we know, they would be moved by it, whereas if we were shown everything they know, we would not find it persuasive of their position. Neither would they, after they had seen what we’ve seen.
I’m talking metaethics, what makes something moral, what it means for something to be moral. Failed ones include divine command theory, the “whatever contributes to human flourishing” idea, whatever makes people happy, whatever matches some platonic ideals out there somehow, whatever leads to selfish interest, etc.
That doesn’t seem obvious to me at all.
Let’s try it on gay marriage. Romans certainly knew and practiced homosexuality, same for marriage. What knowledge exactly do you want to convey to them to persuade them that gay marriage is a good thing?
So, prescriptive. I am not sure in which way do you consider the theories “failed”—in the sense that they have not risen to the status of physics meaning being able to empirically prove all their claims? That doesn’t look to be a viable criterion. In the sense of not having taken over the world? I don’t know, the divine command theory is (or, at least, has been) pretty good at that. You probably wouldn’t want a single theory to take over the world, anyway.
Kind of a weird example, but I’ll assume we’re talking about the Praetorian Guard. The Romans seem to have had very little respect for women and for being penetrated. So right off the bat, having them read a lot of women’s minds might change their views. (I’m not sure if I want to classify that as knowledge, though.) They likely also have false beliefs not only about women but about the gods and stable societies. None of this seems like a cure-all, but it does seem extremely promising.
I don’t understand what that means.
You think no male Roman actually knew what women think? The Roman matrons were entirely voiceless?
I think hairyfigment is of the belief that the Romans (and in the most coherent version of his claim you would have to say male and female) were under misconceptions about the nature of male and female minds, and believes that “a sufficiently deep way” would mean correcting all these misconceptions.
My view is that we really can’t say that as things stand. We’d have to know a lot more about the Roman beliefs about the male and female minds, and compare them against what we know to be accurate about male and female minds.
And what evidence do you have that they laboured under such major misconceptions which we successfully overcame?
I was trying to say with my second paragraph that we specifically cannot be sure about that. My first paragraph was simply my best effort at interpreting what I think hairyfigment thinks, not a statement of what I believe to be true.
From my vague recollections I think the idea is worth looking up one way or the other. After all, a massive portion of modern culture is under the impression there are no gender differences and there are other instances of clear major misconceptions I actually can attest to throughout history. But I don’t have any idea with the Romans.
That’s the stupid portion of modern culture, and I’m not sure they actually, um, practice that belief. Here’s a quick suggestion: make competitive sports sex-blind :-/
I don’t think it’s massive, either.
Yes, I think it is coherent.
Ideological Turing test: I think your theory is this: there is some set of values, which we shall call Morals. All humans have somewhat different sets of lower-case morals. When people make moral mistakes, they can be corrected by learning or internalizing some relevant truths (which may of course be different in each case). These truths can convince even actual humans to change their moral values for the better (as opposed to values changing only over generations), as long as these humans honestly and thoroughly consider and internalize the truths. Over historical time, humans have approached closer to true Morals, and we can hope to come yet closer, because we generally collect more and more truths over time.
If you mean you don’t have any evidence for your theory yet, then how or why did you come by this theory? What facts are you trying to explain or predict with it?
Remember that by default, theories with no evidence for them (and no unexplained facts we’re looking for a theory about) shouldn’t even rise to the level of conscious consideration. It’s far, far more likely that if a theory like that comes to mind, it’s for due to motivated reasoning. For example, wanting to claim your morality is better by some objective measure than that of other people, like slavers.
That’s begging the question. Believing that “people are equal” is precisely the moral belief that you hold and ancient Romans didn’t. Not holding slaves is merely one of many results of having that belief; it’s not a separate moral belief.
But why should Romans come to believe that people are equal? What sort of factual knowledge could lead someone to such a belief, despite the usually accepted idea that should cannot be derived from is?
This is an explanation of Yudkowsky’s idea from the metaethics sequence. I’m just trying to make it accessible in language and length with lots of concept handles and examples.
Technically, you could believe that people are equally allowed to be enslaved. All people equal + it’s wrong to make me a slave = it’s wrong to make anyone a slave.
“All men are created equal” emerges from two or more basic principles people are born with. You might say: “Look, you have value, yah? And your loved ones? Would they stop having value if you forgot about them? No? They have value whether or not you know them? How did you conclude they have value? Could that have happened with other people, too? Would you then think they had value? Would they stop having value if you didn’t know them? No? Well, you don’t know them; do they have value?
You take “people I care about have value” (born with it) and combine it with “be consistent” (also born with), and you get “everyone has value.”
That’s the idea in principle, anyway. You take some things people are all born with, and they combine to make the moral insights people can figure out and teach each other, just like we do with math.
In a sense, the ancient Romans did believe this. Anyone who ended up in the same situation—either taken as a war captive or unable to pay their debts—was liable to be sold as a slave. So what makes you think your position is objectively better than theirs?
This assumes without argument that “value” is something people intrinsically have or can have. If instead you view value as value-to-someone, i.e. I value my loved ones, but someone else might not value them, then there is no problem.
And it turns out that yes, most people did not have an intuition that anyone has intrinsic value just by virtue of being human. Most people throughout history assigned value only to ingroup members, to the rich and powerful, and to personally valued individuals. The idea that people are intrinsically valuable is historically very new, still in the minority today globally, and for both these reasons doesn’t seem like an idea everyone should naturally arrive at if they only try to universalize their intuitions a bit.
You realise that’s a reinvention of Kant?