It seems to me that what’s internal about morality is the buy-in, the acceptance that I ought to care about the other fella at all. But much of what remains is external in the sense that the specific rules of morality to which I subject myself are (at least to a large extent) the product of objective reasoning.
David_J._Balan
Karma: 67
I like the idea of a fictional sequence involving a rationality master and students. But I can’t stand the Jeffreyssai character. He’s just so intolerably smug and self-satisfied, very much in the mold of some of the martial arts instructors I had when I was young. More recently I took boxing classes, and the teacher was like Mickey from the Rocky movies. Much better persona; Jeffreyssai should take note.
After I explained “percentile”, he said “One in three hundred”, so I laughed briefly and said “Yes.”
The “Yes” part is fine. The “I laughed briefly” part would be better done away with.
My sister used to be a teacher in a special education school. She would sometimes let some kids do things that other kids weren’t allowed to do; a kid particularly prone to some kind of negative reaction to an otherwise mandatory activity might be allowed not to participate (I don’t recall exactly). When the other kids protested that it wasn’t fair, she would reply: “fair is when everyone gets what they need, not when everyone gets the same.” Not totally satisfactory, but in my mind not totally bogus either. How hungry each person is does have some bearing on what’s a fair division of the pie.
It seems to me like the word “axioms” belongs in here somewhere.
Of course the feeling of love had to evolve, and of course it had to evolve from something that was not love. And of course the value of the love that we feel is not woven into the fabric of the universe; it’s only valuable to us. But it’s still a very happy thing that love exists, and it’s also sort of a lucky thing; it is not woven into the fabric of the universe that intelligent beings (or any beings for that matter) have to have anything that feels as good as love does to us. This luck may or may not be “surprising” in the sense that it may or may not be the case that the evolution of love (or something else that feels as good to the one who feels it) is highly likely conditional on evolving intelligence. I don’t know the actual answer to this, but the point is that I can at least conceive of a sense in which in which the existence of love might be regarded as surprising.
BTW, the same point can be made about the religious (specifically Protestant) origins of the Enlightenment. The Enlightenment wasn’t always there, and it didn’t fall from the sky, so it had to have its origins in something that wasn’t the Enlightenment. To the extent that Protestantism had some attributes that made it fertile soil for the Enlightenment to grow from, great. But that doesn’t make old-timey Protestantism liberal or good, and it certainly doesn’t entitle contemporary* Protestantism to a share of the credit for the Enlightenments’ achievements.
It is for this reason that Robert Aumann, super-smart as he is, should be entirely ignored when he opines about the Palestinian-Israeli conflict.
http://www.overcomingbias.com/2007/07/theyre-telling-.html
It’s clear that there are some questions to which there are (and likely never will be) fully satisfactory answers, and it is also clear that there is nothing to be done about it but to soldier on and do the best you can (see http://www.overcomingbias.com/2007/05/doubting_thomas.html). However, there really are at least a few things that pretty much have to be assumed without further examination. I have in mind the basic moral axioms like the principle that the other guy’s welfare is something that you should be at all concerned about in the first place.
I seem to recall that there is a strand of philosophy that tries to figure out what unproven axioms would be the minimum necessary foundation on which to build up something like “conventional” morality. They felt the need to do this precisely because of the multi-century failure of philosophers to come up with basis for morality that was unarguable “all the way down” to absolute first principles. I don’t know anything about AI, but it sounds like what Eliezer is talking about here has something of the same flavor.
Along the lines of some of the commenters above, it’s surely not telling Eliezer anything he doesn’t already know to say that there are lots of reasons to be scared that a super-smart AI would start doing things we wouldn’t like even without believing that an AI is necessarily a fundamentally malevolent ghost that will wriggle out of whatever restraints we put it in.
Thursday, February 21st at 7:00 pm? Why bother? Surely the singularity thing will have happened by then:)
Eliezer’s reminder that even rationalists are human, and so are subject to human failings such as turning a community into a cult, is welcome. But it’s a big mistake to dismiss explanations such as “Perhaps only people with innately totalitarian tendencies would try to become the world’s authority on everything.” There is a huge degree of heterogeneity across people in every relevant metric, including a tendency toward totalitarianism. I can’t imagine that anyone disputes this. And if the selection process for being in a certain position tends to advantage people with those tendencies, so that they are selected into them, that might well explain a large part of how people in those positions behave.
Why the hating on summer camp? The good ones are wonderful.
The democracy booster probably meant that people with little political power should not be ignored. And that’s not an empty statement; people with little political power are ignored all the time.
I’m pretty out of my depth here, but I’ll echo what some people have said above. Before people started scientifically doing either one, would it have been obvious that a simple model would be very successful at predicting the behavior of, say, subatomic particles but would be very unsuccessful at predicting the weather? That is, it seems like there really are some phenomena where it is more true and others where it is less true that predictions can be generally and successfully made using straightforward intuitive models. It seems like the label “emergent” is just a (useful) label for the stuff where this can’t be done.
I think some theists would say that the “who made God” question is a semantic stop sign, but that this is OK. That is, they would say that they are not capable in probing into the question any further, but that the leaders of their religion (with the help of the sacred texts) are capable of doing so, and they bring back from the other side the answer that the religion is true and everything is OK.
As for liberal democracy, it’s clearly an error to assert without further argument that liberal democracy will solve all future problems. But it is not a mistake to say that it is far and away the most successful thing that humans have ever come up with, and so that it is the best framework in which to try to address future problems.
Of course it’s always hard to know what truth is in situations like this, but there appears to be evidence that the people who were actually in charge of preventing terrorism were actively worried about something much like what actually happened, and were ignored by their superiors.
Eliezer, you shouldn’t have chased Anna away.
I’m very sympathetic to the idea contained in the post. In fact, I used to say something similar in my graduate student gig as a freshman orientation guy. But teaching and learning real thought are hard. Can everybody teach it and learn it? Is there any sense in which what goes on now is not optimal, but is (or at least is not too far from) constrained optimal?
This reminds me of the bit in Steven Landsburg’s (excellent) book “The Armchair Economist” in which he makes the point that data on what happens on third down in football games is a very poor guide to what would happen on third down if you eliminated fourth down.