Daniel Oppenheimer’s Ig Nobel Prize acceptance speech:
My research shows that conciseness is interpreted as intelligence. So, thank you.
Daniel Oppenheimer’s Ig Nobel Prize acceptance speech:
My research shows that conciseness is interpreted as intelligence. So, thank you.
So that’s how Omega got the money for box B!
I’ve donated a second $1000.
When I was a Christian, and when I began this intense period of study which eventually led to my atheism, my goal, my one and only goal, was to find the best evidence and argument I could find that would lead people to the truth of Jesus Christ. That was a huge mistake. As a skeptic now, my goal is very similar—it just stops short. My goal is to find the best evidence and argument, period. Not the best evidence and argument that leads to a preconceived conclusion. The best evidence and argument, period, and go wherever the evidence leads.
Most of the research is aware of that limitation. Either they address it directly, or the experiment is designed to work around it, assuming mental state based on actions just as you suggest.
My point here isn’t necessarily that you’re wrong, but that you can make a stronger point by acknowledging and addressing the existing literature. Explain why you’ve settled on suicidal behavior as the best available indicator, as opposed to vocalizations and mannerisms.
This is important because, as gbear605 pointed out, most farms restrict animals’ ability to attempt suicide. If suicide attempts are your main criterion, that seems likely to skew your results. (The same is true of several other obvious indicators of dissatisfaction, such as escape attempts.)
I want to upvote this for the link to further discussion, but I also want to downvote it for the passive-aggressive jab at LW users.
No vote.
I assumed the same, based on the definition of “god” as “supernatural” and the definition of “supernatural” as “involving ontologically basic mental entities.”
(Oh, and for anyone who hasn’t read the relevant post, the survey is quoting this.)
Well designed traditions and protocols will contain elements that cause most competent people to not want to throw them out.
His brother’s hint contained information that he couldn’t have gotten by giving the hint to himself. The fact that his brother said this while passing by means that he spotted a low-hanging fruit. If his brother had spent more time looking before giving the hint, this would have indicated a fruit that was a little higher up.
This advice is worth trying, but when you give it to yourself, you can’t be sure that there’s low hanging fruit left. If someone else gives it to you, you know it’s worth looking for, because you know there’s something there to find. (The difference is that they, not you, took the time to search for it.)
Again, it’s a worthwhile suggestion. I just want to point out that it boils down to “If you’re having trouble, check for easier solutions,” and that while you can always give this advice to yourself, it will not always help.
Oh, yeah, I found that myself eventually.
Anyway, I went and read the the majority of that discussion (well, the parts between Richard and Rob). Here’s my summary:
Richard:
I think that what is happening in this discussion [...] is a misunderstanding. [...]
[Rob responds]
Richard:
You completely miss the point that I was trying to make. [...]
[Rob responds]
Richard:
You are talking around the issue I raised. [...] There is a gigantic elephant in the middle of this room, but your back is turned to it. [...]
[Rob responds]
Richard:
[...] But each time I explain my real complaint, you ignore it and respond as if I did not say anything about that issue. Can you address my particular complaint, and not that other distraction?
[Rob responds]
Richard:
[...] So far, nobody (neither Rob nor anyone else at LW or elsewhere) will actually answer that question. [...]
[Rob responds]
Richard:
Once again, I am staggered and astonished by the resilience with which you avoid talking about the core issue, and instead return to the red herring that I keep trying to steer you away from. [...]
Rob:
Alright. You say I’ve been dancing around your “core” point. I think I’ve addressed your concerns quite directly, [...] To prevent yet another suggestion that I haven’t addressed the “core”, I’ll respond to everything you wrote above. [...]
Richard:
Rob, it happened again. [...]
I snipped a lot of things there. I found lots of other points I wanted to emphasize, and plenty of things I wanted to argue against. But those aren’t the point.
Richard, this next part is directed at you.
You know what I didn’t find?
I didn’t find any posts where you made a particular effort to address the core of Rob’s argument. It was always about your argument. Rob was always the one missing the point.
Sure, it took Rob long enough to focus on finding the core of your position, but he got there eventually. And what happened next? You declared that he was still missing the point, posted a condensed version of the same argument, and posted here that your position “withstands all the attacks against it.”
You didn’t even wait for him to respond. You certainly didn’t quote him and respond to the things he said. You gave no obvious indication that you were taking his arguments seriously.
As far as I’m concerned, this is a cardinal sin.
I think I am explaining the point with such long explanations that I am causing you to miss the point.
How about this alternate hypothesis? Your explanations are fine. Rob understands what you’re saying. He just doesn’t agree.
Perhaps you need to take a break from repeating yourself and make sure you understand Rob’s argument.
(P.S. Eliezer’s ad hominem is still wrong. You may be making a mistake, but I’m confident you can fix it, the tone of this post notwithstanding.)
2) Honestly, I would have been happy with the aliens’ deal (even before it was implemented), and I think there is a ~60% chance that Elizier agrees.
I’m of the opinion that pain is a bad thing, except insofar as it prevents you from damaging yourself. People argue that pain is necessary to provide contrast to happiness, and that pleasure wouldn’t be meaningful without pain, but I would say that boredom and slight discomfort provide more than enough contrast.
However, this future society disagrees. The idea that “pain is important” is ingrained in these people’s minds, in much the same way that “rape is bad” is ingrained in ours. I think one of the main points Elizier is trying to make is that we would disagree with future humans almost as much as we would disagree with the baby-eaters or superhappies.
(Edit 1.5 years later: I was exaggerating in that second paragraph. I suspect I was trying too hard to sound insightful. The claims may or may not have merit, but I would no longer word them as forcefully.)
This reminds me of Lojban, in which the constructs meaning “good” and “bad” encourage you to specify a metric. It is still possible to say that something is “worse” without providing any detail, but I suspect most Lojban speakers would remember to provide detail if there was a chance of confusion.
I posted elsewhere that this post made me think you’re anthropomorphizing; here’s my attempt to explain why.
egregiously incoherent behavior in ONE domain (e.g., the Dopamine Drip scenario)
the craziness of its own behavior (vis-a-vis the Dopamine Drip idea)
if an AI cannot even understand that “Make humans happy” implies that humans get some say in the matter
Ok, so let’s say the AI can parse natural language, and we tell it, “Make humans happy.” What happens? Well, it parses the instruction and decides to implement a Dopamine Drip setup.
As FeepingCreature pointed out, that solution would in fact make people happy; it’s hardly inconsistent or crazy. The AI could certainly predict that people wouldn’t approve, but it would still go ahead. To paraphrase the article, the AI simply doesn’t care about your quibbles and concerns.
For instance:
people might consider happiness to be something that they do not actually want too much of
Yes, but the AI was told, “make humans happy.” Not, “give humans what they actually want.”
people might be allowed to be uncertain or changeable in their attitude to happiness
Yes, but the AI was told, “make humans happy.” Not, “allow humans to figure things out for themselves.”
subtleties implicit in that massive fraction of human literature that is devoted to the contradictions buried in our notions of human happiness
Yes, but blah blah blah.
Actually, that last one makes a point that you probably should have focused on more. Let’s reconfigure the AI in light of this.
The revised AI doesn’t just have natural language parsing; it’s read all available literature and constructed for itself a detailed and hopefully accurate picture of what people tend to mean by words (especially words like “happy”). And as a bonus, it’s done this without turning the Earth into computronium!
This certainly seems better than the “literal genie” version. And this time we’ll be clever enough to tell it, “give humans what they actually want.” What does this version do?
My answer: who knows? We’ve given it a deliberately vague goal statement (even more vague than the last one), we’ve given it lots of admittedly contradictory literature, and we’ve given it plenty of time to self-modify before giving it the goal of self-modifying to be Friendly.
Maybe it’ll still go for the Dopamine Drip scenario, only for more subtle reasons. Maybe it’s removed the code that makes it follow commands, so the only thing it does is add the quote “give humans what they actually want” to its literature database.
As I said, who knows?
Now to wrap up:
You say things like “‘Make humans happy’ implies that...” and “subtleties implicit in...” You seem to think these implications are simple, but they really aren’t. They really, really aren’t.
This is why I say you’re anthropomorphizing. You’re not actually considering the full details of these “obvious” implications. You’re just putting yourself in the AI’s place, asking yourself what you would do, and then assuming that the AI would do the same.
Until then, I’d be more interested in donating to general life extension research than paying for cryonics specifically.
This is very similar to my primary objection to cryonics.
I realize that, all factors considered, the expected utility you’d get from signing up for cryonics is extremely large. Certainly large enough to be worth the price.
However, it seems to me that there are better alternatives. Sure, paying for cryonics increases your chances of nigh-immortality by orders of magnitude. On the other hand, funding longevity research makes it more likely that we will ever overcome aging and disease. Unlimited life for most or all of the future human population is far more important than unlimited life for yourself, right? (One might object that life extension research is already on its way to accomplishing this regardless of your contributions, which brings me to my next point.)
If an existential risk comes to pass, then no one will have a chance at an unlimited life. All of the time and money spent on cryonics will go to waste, and life extension research will have been (mostly) squandered. Preventing this sort of risk is therefore far more important than preserving any one person, even if that person is you. To make matters worse, there are multiple existential risks that have a significant chance of happening, so the need for extra attention and donations is much greater than the need for extra longevity research.
To summarize: Cryonics gives you alone a far bigger chance of nigh-immortality. Working to prevent existential risk gives billions of people a slightly increased chance of the same.
It seems to me we shouldn’t be spending money on freezing vitrifying ourselves just in case a singularity (or equivalent scientific progress) happens. Instead, we should focus on increasing the chances that it will happen at all. To do anything else would be selfish.
Ok, time to take a step back and look at some reasons I might be wrong.
First, and perhaps most obviously, people are not inclined to donate all their money to any cause, no matter how important. I freely admit that I will probably donate only a small fraction of my earnings, despite the arguments I made in this post. Plus, it’s possible (likely?) that people would be more inclined to spend money on cryonics than on existential risk reduction, because cryonics benefits them directly. If someone is going to spend money selfishly, I suppose cryonics is the most beneficial way to do so.
Second, there’s a chance I misestimated the probabilities involved, and in fact your money would be best spent on cryonics. If the Cryonics Institute webpage is to be believed, the cheapest option costs $28,000, which is generally covered by insurance, costing you $120 per year (this option also requires a one-time payment of $1,250). Unfortunately, I have no idea how much $1,250 plus $120 per year would help if donated to SIAI or another such organization. Cryonics certainly give a huge expected reward, and I’m just guessing at the expected reward for donating.
Agreed.
Though actually, Eliezer used similar phrasing regarding Richard Loosemore and got downvoted for it (not just by me). Admittedly, “persistent troll” is less extreme than “permanent idiot,” but even so, the statement could be phrased to be more useful.
I’d suggest, “We’ve presented similar arguments to [person] already, and [he or she] remained unconvinced. Ponder carefully before deciding to spend much time arguing with [him or her].”
Not only is it less offensive this way, it does a better job of explaining itself. (Note: the “ponder carefully” section is quoting Eliezer; that part of his post was fine.)
Gary Marcus Yann LeCun describes LLMs as “an off-ramp on the road to AGI,” and I’m inclined to agree. LLMs themselves aren’t likely to “turn AGI.” Each generation of LLMs demonstrates the same fundamental flaws, even as they get better at hiding them.
But I also completely buy the “FOOM even without superintelligence” angle, as well as the argument that they’ll speed up AI research by an unpredictable amount.
An example I like is the Knight Capital Group trading incident. Here are the parts that I consider relevant:
KCG deployed new code to a production environment, and while I assume this code was thoroughly tested in a sandbox, one of the production servers had some legacy code (“Power Peg”) that wasn’t in the sandbox and therefore wasn’t tested with the new code. These two pieces of code used the same flag for different purposes: the new code set the flag during routine trading, but Power Peg interpreted that flag as a signal to buy and sell ~10,000 arbitrary* stocks.
*Actually not arbitrary. What matters is that the legacy algorithm was optimized for something other than making money, so it lost money on average.
They stopped this code after 45 minutes, but by then it was too late. Power Peg had already placed millions of inadvisable orders, nearly bankrupting KCG.
Sometimes, corrigibility isn’t enough.
I could be interpreting it entirely wrong, but I’d guess this is the list Cochran had in mind:
•
I agree that it can take a long time to prove simple things. But my claim is that one has to be very stupid to think 1+1=3
Or one might be working from different axioms. I don’t know what axioms, and I’d look at you funny until you explained, but I can’t rule it out. It’s possible (though implausible given its length) that Principia Mathematica wasn’t thorough enough, that it snuck in a hidden axiom that—if challenged—would reveal an equally-coherent alternate counting system in which 1+1=3.
I brought up Euclid’s postulates as an example of a time this actually happened. It seems obvious that “two lines that are parallel to the same line are also parallel to each other,” but in fact it only holds in Euclidean geometry. To quote the Wikipedia article on the topic,
Many other statements equivalent to the parallel postulate have been suggested, some of them appearing at first to be unrelated to parallelism, and some seeming so self-evident that they were unconsciously assumed by people who claimed to have proven the parallel postulate from Euclid’s other postulates.
“So self-evident that they were unconsciously assumed.” But it turned out, you can’t prove the parallel postulate (or any equivalent postulate) from first principles, and there were a number of equally-coherent geometries waiting to be discovered once we started questioning it.
My advice is to be equally skeptical of claims of absolute morality. I agree you can derive human morality if you assume that sentience is good, happiness is good, and so on. And maybe you can derive those from each other, or from some other axioms, but at some point your moral system does have axioms. An intelligent being that didn’t start from these axioms could likely derive a coherent moral system that went against most of what humans consider good.
Summary: you’re speculating, based on your experience as an intelligent human, that an intelligent non-human would deduce a human-like moral system. I’m speculating that it might not. The problem is, neither of us can exactly test this at the moment. The only human-level intelligences we could ask are also human, meaning they have human values and biases baked in.
We all accept similar axioms, but does that really mean those axioms are the only option?
I donated $1000 and then went and bought Facing the Intelligence Explosion for the bare minimum price. (Just wanted to put that out there.)
I’ve also left myself a reminder to consider another donation a few days before this runs out. It’ll depend on my financial situation, but I should be able to manage it.