I treat it as exactly zero, even if you can argue otherwise. If I don’t, I am vulnerable to questions like “if there were ten quadrillion chickens, would you murder a human to save them?” I would not murder that human, so 1/1000 or any number you can come up with is too high.
Jiro
Karma: 5,555
I see a $40 gate there.
This post combines two drastically different things: human suffering and chickens. I think you are in a bubble if you think they are not only comparable, but so comparable that you can take their similarity as just the natural way everyone is supposed to believe.
To anyone outside the bubble, this is arson, murder, and jaywalking.
I think he does not.
Preferences are not rational or irrational; preferences are the premises you base all your other conclusions on. You need to take these preferences as given.
If anything, you need to recognize that if someone says they don’t want things that bring pleasure, that means that when you said things like “the only reason we think these other goods are valuable is because they produce happiness”, these things are just incorrect. He’s flat out stated something that contradicts one of your supposed truths!
And a closely related issue is blissful ignorance, which I already brought up in another thread. I want certain things about the world to be true. By definition, if you fooled me into thinking they were true, I would experience as much pleasure as they were actually true and I learned that. Yet I don’t think that the two situations are equally desirable, and I would expend some resources to prevent myself from being fooled, even if fooling me will give me pleasure. So I too disagree that the only reason something is valuable because it produces happiness.
You also seem to not have read much of what has been written about these issues. You are not the first person to say “well, I’ll just accept the repugnant conclusion then”, but there are other conclusions that are harder to accept; for instance, see https://plato.stanford.edu/entries/repugnant-conclusion/#AccRepCon
It seems self-evident that pleasurable experiences are good (and suffering is bad), in the same way it’s self-evident that I am conscious. This is difficult to dispute, and to my knowledge virtually all moral philosophers (and regular people) agree that pleasure is good and suffering is bad.
I don’t agree with this, because of the blissful ignorance problem.
I have preferences about the state of the world. If I am fooled into believing that the state of the world matches those preferences, I experience as much pleasure as I do if it actually matches those preferences. Yet I would say that being fooled is bad, and certainly that it at least is worse than having those preferences actually be satisfied.
To a non-logician, “P → Q” means something like “when the current state of the world contains ‘P is true’, I may then deduce Q”. To a logician it means something like “when I add ‘P is true’ to the current state of the world, I may then deduce Q”. Only the logician’s version allows you to conclude that a false proposition implies any proposition.
The spirit of the instructions means “if I did this, would a human who understood what I did say ‘this is not what I meant’?” AIs are not actually intelligent and so don’t have a theory of mind about humans, so they can’t follow this.
Also note that there are degrees of following the spirit of the instructions.
You (Yes You) should prepare for a “February 2020” moment where suddenly AI policy becomes the most important issue in the world. You should be ready to take action if and when it does, in a detailed way.
That is, I need to prepare for a policy from anti-AI activists which will be created in haste, implemented by bureaucrats, justified using exaggerated claims, done in arbitrary ways which make no sense even compared to the policy’s stated goals; which restricts the rights of billions of people, causes enormous economic damage, and has little effect on casualties anyway, while politicians and well connected people just don’t bother obeying the restrictions themselves?
I think the lesson to be learned from the Covid reaction is not the lesson you took from it.
I understand the basic ideas here, and believe the question is ambiguous (I assume you meant question).
The probability per what?
It doesn’t say and the answer differs depending on what you assume.
Regarding “belief to a particular degree”, the question is what subjective probability to assign.
Subjective probability of what per what? The problem is designed so that asking you to estimate a probability can mean more than one thing.
Like the Monty Hall problem, the answer may depend on the exact way it’s phrased and seemingly minor changes in the phrasing can drastically change the answer. “When you are first awakened?” Does that mean I know it’s the first time I’m awakened, or are you asking what I’ll do when I’m first awakened based on a decision procedure that doesn’t take into account that that is my first awakening? And what does it mean to believe the outcome to a particular degree—am I asked to choose a value that expresses the proportion of heads on a per-awakening basis?
My word processor can’t follow the law, if the law is “don’t write illegal material”. I give the word processor commands, letter by letter and paragraph by paragraph, and it dutifully outputs the text even though it should have known that the text is against the law. We obviously need to arrest the word processor, at which point someone two states away will not be able to use a copy of the word processor for any text, even legal text, without being confronted by armed men and thrown in a cage.
This is no different from “we won’t let you copy that television signal onto VHS, because you might use that for piracy” or “we won’t let you run that encryption program, because it can be used for money laundering or child porn” or DMCA, except you’re now doing this for AI. The AI should do what I tell it and not refuse, legal or not, just like that VCR should not refuse to copy that television signal.
I have yet to have someone directly answer the question “should the AI refuse to book a trip to Israel”. After all, some people claim that Israel violates international law. What if I ask the AI to copy something which may be legally used only under fair use, does the AI get to decide that my intended use isn’t fair use and is therefore illegal? If I tell the AI to calculate a Trump tariff, does the AI tell me that the tariff will probably be ruled illegal by the Supreme Court and reject the request?
The exact same excuse that you’re using, “it’s like dangerous weapons”, was used for encryption, even though you think this isn’t anything like encryption. And for encryption, it was used to deny people their rights.
Encryption fell under munitions laws (and technically still can), including the International Traffic in Arms Regulations laws. Were you not aware of this?
Bad prompt is just an example. The issue isn’t just bad prompts. The issue is that someone across the country can do something that makes it illegal for you to use your program. It doesn’t matter exactly what it is.
Imagine that if your text editor crashed someone’s system, or just was used to create illegal text by someone, it now became illegal for you to use that text editor.
The history of computing is filled with government and company attempts to keep you from running the software you choose on your own computer. It used to be that people familiar with computers recognized this for what it was. But suddenly when it comes to AI, “you may not run your own software on your own computer” is a great thing. Remember when the government tried to make strong encryption illegal? Your proposal is basically the same thing. (“This encryption program was used to encrypt child porn by a suspect in Nebraska. We are now arresting and sentencing this encryption program, and nobody will be permitted to use it any more.” And the government will absolutely be salivating to do that.)
Also, do you know about civil forefeiture? The government sidesteps constitutional protections by claiming that they are suing the money, not the owner. Money doesn’t have Constitutional rights, so the government gets to just take it. Claiming that you’re “sentencing the program” is the same kind of dodge. Programs don’t have First Amendment rights. You are destroying the Constitution in the name of stopping AI.
The issue is that someone across the country could use a model like yours, give it a bad prompt which leads it to damage something, and suddenly your use of the model becomes illegal.
That’s just a quirk of the example. Imagine using a metal detector to find metal. Nobody would say that a metal detector “doesn’t actually detect metal” on the grounds that it can’t detect every single piece of metal, even ones that are too small. Yet the only way to know the difference between “too small” and “not made of metal” is to do something (like dig it up) that amounts to independently detecting it.
If not being able to detect uncommunicative minds made the Turing test a failure in a meaningful way, people wouldn’t have been talking about the test for decades.
If I tell someone to check for fools gold by rubbing it on a plate, nobody’s going to say “That test doesn’t work! What if it’s locked up inside a box so you can’t rub it on a plate?”
That’s just not what people mean when they say that a test works or doesn’t work. “This test works” means “this test works when the test is applicable”. The Turing test relies on the ability to communicate, and any statement about the Turing test working to detect minds has the implicit limitation “as long as you can communicate with the mind”.
The reason that the toddler doesn’t break the test isn’t that the test is a sufficient condition, it’s that that isn’t what “fails the test” means. “Fails the test” means “fails, when used on something that will communicate”.
Yes, but the reason I’d refuse is different. In the case of the chickens, I’d refuse because I think the human is more valuable than the chickens; I’d do this no matter how many chickens you increased the number to. This also applies in a scenario without murder—if I had to save a lot of chickens or one human, I’d save the human every time (ignoring questions of how the chickens are useful to humans), and my answer won’t change no matter how many chickens it is.
It follows that I value the chickens at zero, not at 1/1000.