@Eliezer I mostly agree with Caledonian here. I disagree with much of what you say, and it has nothing to do with being ‘fooled’. Censoring the few dissenters who actually comment is not a good idea if you have any interest in avoiding an echo chamber. You’re already giving off the Louis Savain vibe pretty hard.
Thom_Blake
You’re forgetting the philosopher’s dotted-line trick of making it clearer by saying it in a foreign language. “Oh, you thought I meant ‘happiness’ which is ill-defined? I actually meant ‘eudaimonia’!”
But if there are repeatable psi experiments, then why hasn’t anyone won the million dollars? (or even passed the relatively easy first round?)
Several places in the US did have regulations protecting the horse industry from the early automobile industry—I’m not sure what “the present system” refers to as opposed to that sort of thing.
Avast! But ‘ought’ ain’t needin’ to be comin’ from another ‘ought’, if it be arrived at empirically. Yar.
Fer a bit thar I were thinkin’ that ye’d be agreein’ with that yellow-bellied skallywag Hanson. Yar, but the Popperians ha’ it! A pint of rum fer ol’ Eliezer!
scott clark,
I think your history is a bit off. The plan wasn’t ‘originally’ for Luke to kill Vader, his father; it wasn’t until midway through filming Empire (or at least, after the release of A New Hope) that Lucas decided that Vader was Luke’s father.
I think if history remembers you, I’d bet that it will be for the journey more than its end. If the interesting introspective bits get published in a form that gets read, then I’d bet it will be memorable in the way that Lao zi or Sun zi is memorable. In case the Singularity / Friendly AI stuff doesn’t work out, please keep up the good work anyway.
This doesn’t seem to mesh with the Friendly AI goal of getting it perfectly right on the first try.
Do we accept some uncertainty and risk to do something extraordinary now, or do we take the slow, calm, deliberative course that stands a chance of achieving perfection?
Is there any chance of becoming a master of the blade without beginning to cut?
I’m with Kaj on this. Playing the AI, one must start with the assumption that there’s a rock on the “don’t let the AI out” button. That’s why this problem is impossible. I have some ideas about how to argue with ‘a rock’, but I agree with the sentiment of not telling.
Russell, I don’t think that necessarily specifies a ‘cheap trick’. If you start with a rock on the “don’t let the AI out” button, then the AI needs to start by convincing the gatekeeper to take the rock off the button. “This game has serious consequences and so you should really play rather than just saying ‘no’ repeatedly” seems to be a move in that direction that keeps with the spirit of the protocol, and is close to Silas’s suggestion.
“Everyone has a right to their own opinion” is largely a product of its opposite. For a long period many people believed “If my neighbor has a different opinion than I do, then I should kill him”. This led to a bad state of affairs and, by force, a less lethal meme took hold.
pdf, the only reason that suggestion works is that we’re not in the business of bombing headquarters at 2AM on a weekend. If both sides were scheduling bombings at 2AM, I’d bet they’d be at work at 2AM.
Toby,
You should never, ever murder an innocent person who’s helped you, even if it’s the right thing to do
You should never, ever do X, even if if you are exceedingly confident that it is the right thing to do
I believe a more sensible interpretation would be, “You should have an unbreakable prohibition against doing X, even in cases where X is the right thing to do”—the issue is not that you might be wrong about it being the right thing to do, but rather that not having the prohibition is a bad thing.
Anonymous: torture’s inefficacy was well-known by the fourteenth century; Bernardo Gui, a famous inquisitor who supervised many tortures, argued against using it because it is only good at getting the tortured to say whatever will end the torture. I can’t seem to find the citation, but here is someone who refers to it: http://www.ewtn.com/library/ANSWERS/INQUIS2.htm
I’ve yet to hear of anyone turning back successfully, though I think some have tried, or wished they could.
It seems to be one interpretation of the Buddhist project
Regarding self, I tend to include much more than my brain in “I”—but then, I’m not one of those who thinks being ‘uploaded’ makes a whole lot of sense.
Interesting idea… though I still think you’re wrong to step away from anthropomorphism, and ‘necessary and sufficient’ is a phrase that should probably be corralled into the domain of formal logic.
And I’m not sure this adds anything to Sternberg and Salter’s definition: ‘goal-directed adaptive behavior’.
Tilden is another roboticist who’s gotten rich and famous off of unintelligent robots: BEAM robotics
Jed, your comment (the second example, specifically) reminds me of the story about how the structure of DNA was discovered. Apparently the ‘Eureka’ moment actually came after the researchers obtained better materials for modeling.
Aristotelians may not be teaching physics courses (though I know of no survey showing that) but they do increasingly teach ethics courses. It makes sense to think of what qualities are good for a fox or good for a rabbit, and so one can speak about them with respect to ethics.
However, there is no reason to think that they disagree about ethics, since disagreement is a social activity that is seldom shared between species, and ethics requires actually thinking about what one has most reason to do or want. While it makes sense to attribute the intentional stance to such animals to predict their behavior (a la Dennett), that still leaves us with little reason to regard them as things that reason about morality.
That said, there is good reason to think that dogs, for instance, have disagreements about ethics. Dogs have a good sense of what it takes to be a good dog, and will correct each other’s behavior if it falls out of line (as mentioned w.r.t. wolves above).