At age 5 you could safely wish for “I wish for you to do what I should wish for” and at worst you’d be a little disappointed if what she came up with wasn’t as fun as you’d have liked.
I would have gotten the wrong flavor of ice cream. It was strictly better to specify the flavor of ice cream I preferred. Therefore, the statement about the 3 types of genies is simply false. It might be approximately true in some sense, but even if it is, the article never gives any arguments in favor of that thesis, it simply gives one example.
Wait, to be clear, you’re calling getting the wrong flavor of icecream a “safety” issue?
Do you have any examples that actually fall outside the 3 types? Your mother is likely not powerful and nor is she a superintelligence. So far the only example you’ve given has fallen squarely in the third category but even if scaled up would probably fit quite well in the first.
I’d also note that the claim you’re taking issue with is a metaphor for explaining things, he’s not claiming that magical genies actually exist of any category.
His metaphor completely fails conceptually, because I’m perfectly capable of imagining genies that fall outside the three categories.
Perhaps the classification works in some other setting, such as AIs. However, the article never provided any arguments for this (or any arguments at all, really). Instead, there was one single example (seriously, just one example!) which was then extrapolated to all genies.
Examples of what? Of hypothetical intelligent minds? I feel like there are examples all over fiction; consider genies themselves, which often grant wishes in a dangerous way (but you can sometimes get around it by speaking carefully enough). Again, I agree that some genies are never safe and some are always safe, but it’s easy to imagine a genie which is safe if and only if you specify your wish carefully.
Anyway, do you concede the point that EY’s article contains no arguments?
If you have to speak “carefully enough” then you’re taking a big risk though you may luck out and get what you want, they’re not safe.
EY’s article contains arguments, you just seem to have picked up on something that wasn’t what he was arguing about.
It’s like someone started a speech with “Good evening ladies and gentlemen.” and your criticism was that he failed to prove that it was evening, failed to prove that there was a mix of genders in the audience and that the entirety of the rest of the speech failed to contain any arguments about whether the men in the audience were in fact gentlemen.
It contained a very clear and well made argument for why simply trying to word your wish carefully was a fools errand.
You may notice how it starts with an overly complex wish from the “open source wish project”. It then gives examples of how simply adding clauses to the wish to get your mother out doesn’t help much because you value so many things as a human that you’d have to add so many thousands of disclaimers, clauses and rules that it would be insane while missing even one could mean disaster(from your point of view) which is extremely unsafe.
If you have to speak “carefully enough” then you’re taking a big risk though you may luck out and get what you want, they’re not safe.
If your argument is that unless a powerful being is extremely safe, then they’re not extremely safe, this is true by definition. Obviously, if a genie sometimes doesn’t give you what you want, there is some risk that the genie won’t give you what you want. I thought a more substantial argument was being made, though—it sounded like EY was claiming that saying “I wish for whatever I should wish for” is supposed to always be better than every other wish. This claim is certainly false, due to the “mom” example. So I guess I’m left being unsure what the point is.
I see you’ve not bothered reading any of my replies and instead just made up your own version in your head.
Your mom example falls quite cleanly into the third catagory if it doesn’t fall cleanly into the first.
Unless a powerful being understands your values well enough to take them into account and actually wants to take them into account then it’s not extremely safe. Yes.
Believe it or not there are a lot of people who’ll do things like insist that that’s not the case or insist that you just have to wish carefully enough hence the need for the article.
I see you’ve not bothered reading any of my replies and instead just made up your own version in your head.
I read all of your replies. What are you referring to? Also, this is uncharitable/insulting.
Believe it or not there are a lot of people who’ll do things like insist that that’s not the case or insist that you just have to wish carefully enough hence the need for the article.
To be honest, I’m not sure what we’re even disagreeing about. Like, sure, some genies are unsafe no matter how you phrase your wish. For other genies, you can just wish for “whatever I ought to wish for”. For still other genies, giving some information about your wish helps.
If EY’s point was that the first type of genies exist, then yes, he’s made it convincingly. If his point is that you never need to specify a wish other than “whatever I ought to wish for” (assuming a genie is powerful enough), then he failed to provide arguments for this claim (and the claim is probably false).
I would have gotten the wrong flavor of ice cream. It was strictly better to specify the flavor of ice cream I preferred. Therefore, the statement about the 3 types of genies is simply false. It might be approximately true in some sense, but even if it is, the article never gives any arguments in favor of that thesis, it simply gives one example.
Wait, to be clear, you’re calling getting the wrong flavor of icecream a “safety” issue?
Do you have any examples that actually fall outside the 3 types? Your mother is likely not powerful and nor is she a superintelligence. So far the only example you’ve given has fallen squarely in the third category but even if scaled up would probably fit quite well in the first.
I’d also note that the claim you’re taking issue with is a metaphor for explaining things, he’s not claiming that magical genies actually exist of any category.
I’m making 2 points:
His metaphor completely fails conceptually, because I’m perfectly capable of imagining genies that fall outside the three categories.
Perhaps the classification works in some other setting, such as AIs. However, the article never provided any arguments for this (or any arguments at all, really). Instead, there was one single example (seriously, just one example!) which was then extrapolated to all genies.
Ok, so, do you actually have any examples that fall outside the 3 categories:
1:Powerful+Safe
2:Powerful+Unsafe
3:Not very powerful such that it doesn’t matter so much if they’re safe or unsafe.
Examples of what? Of hypothetical intelligent minds? I feel like there are examples all over fiction; consider genies themselves, which often grant wishes in a dangerous way (but you can sometimes get around it by speaking carefully enough). Again, I agree that some genies are never safe and some are always safe, but it’s easy to imagine a genie which is safe if and only if you specify your wish carefully.
Anyway, do you concede the point that EY’s article contains no arguments?
If you have to speak “carefully enough” then you’re taking a big risk though you may luck out and get what you want, they’re not safe.
EY’s article contains arguments, you just seem to have picked up on something that wasn’t what he was arguing about.
It’s like someone started a speech with “Good evening ladies and gentlemen.” and your criticism was that he failed to prove that it was evening, failed to prove that there was a mix of genders in the audience and that the entirety of the rest of the speech failed to contain any arguments about whether the men in the audience were in fact gentlemen.
It contained a very clear and well made argument for why simply trying to word your wish carefully was a fools errand.
You may notice how it starts with an overly complex wish from the “open source wish project”. It then gives examples of how simply adding clauses to the wish to get your mother out doesn’t help much because you value so many things as a human that you’d have to add so many thousands of disclaimers, clauses and rules that it would be insane while missing even one could mean disaster(from your point of view) which is extremely unsafe.
If your argument is that unless a powerful being is extremely safe, then they’re not extremely safe, this is true by definition. Obviously, if a genie sometimes doesn’t give you what you want, there is some risk that the genie won’t give you what you want. I thought a more substantial argument was being made, though—it sounded like EY was claiming that saying “I wish for whatever I should wish for” is supposed to always be better than every other wish. This claim is certainly false, due to the “mom” example. So I guess I’m left being unsure what the point is.
I see you’ve not bothered reading any of my replies and instead just made up your own version in your head.
Your mom example falls quite cleanly into the third catagory if it doesn’t fall cleanly into the first.
Unless a powerful being understands your values well enough to take them into account and actually wants to take them into account then it’s not extremely safe. Yes.
Believe it or not there are a lot of people who’ll do things like insist that that’s not the case or insist that you just have to wish carefully enough hence the need for the article.
I read all of your replies. What are you referring to? Also, this is uncharitable/insulting.
To be honest, I’m not sure what we’re even disagreeing about. Like, sure, some genies are unsafe no matter how you phrase your wish. For other genies, you can just wish for “whatever I ought to wish for”. For still other genies, giving some information about your wish helps.
If EY’s point was that the first type of genies exist, then yes, he’s made it convincingly. If his point is that you never need to specify a wish other than “whatever I ought to wish for” (assuming a genie is powerful enough), then he failed to provide arguments for this claim (and the claim is probably false).