Questionable. How is an encapsulated AI going to get this kind of control without already existing advanced nanotechnology? It might order something over the Internet if it hacks some bank account etc. (long chain of assumptions),
Any specific scenario is going to have burdensome details, but that’s what you get if you ask for specific scenarios rather than general pressures, unless one spends a lot of time going through detailed possibilities and vulnerabilities. With respect to the specific example, regular human criminals routinely swindle or earn money anonymously online, and hack into and control millions of computers in botnets. Cloud computing resources can be rented with ill-gotten money.
but how is it going to make use of the things it orders?
In the unlikely event of a powerful human-indifferent AI appearing in the present day, a smartphone held by a human could provide sensors and communication to use humans for manipulators (as computer programs direct the movements of some warehouse workers today). Humans can be paid, blackmailed, deceived (intelligence agencies regularly do these things) to perform some tasks. An AI that leverages initial capabilities could jury-rig a computer-controlled method of coercion [e.g. a cheap robot arm holding a gun, a tampered-with electronic drug-dispensing implant, etc]. And as time goes by and the cumulative probability of advanced AI becomes larger, increasing quantities of robotic vehicles and devices will be available.
Thanks, yes I know about those arguments. One of the reasons I’m actually donating and accept AI to be one existential risk. I’m inquiring about further supporting documents and transparency. More on that here, especially check the particle collider analogy.
With respect to transparency, I agree about a lack of concise, exhaustive, accessible treatments. Reading some of the linked comments about marginal evidence from hypotheses I’m not quite sure what you mean, beyond remembering and multiplying by the probability that particular premises are false. Consider Hanson’s “Economic Growth Given Machine Intelligence”. One might support it with generalizations from past population growth in plants and animals, from data on capital investment and past market behavior and automation, but what would you say would license drawing probabilistic inferences using it?
Note that such methods might not result in the destruction of the world within a week (the guaranteed result of a superhuman non-Friendly AI according to Eliezer.)
The linked bet doesn’t reference “a week,” and the “week” reference in the main linked post is about going from infrahuman to superhuman, not using that intelligence to destroy humanity.
That bet seems underspecified. Does attention to “Friendliness” mean any attention to safety whatsoever, or designing an AI with a utility function such that it’s trustworthy regardless of power levels? Is “superhuman” defined relative to the then-current level of human (or upload, or trustworthy less intelligent AI) capacity with any enhancements (or upload speedups, etc)? What level of ability counts as superhuman? You two should publicly clarify the terms.
A few comments later on the same comment thread someone asked me how much time was necessary, and I said I thought a week was enough, based on Eliezer’s previous statements, and he never contradicted this, so it seems to me that he accepted it by default, since some time limit will be necessary in order for someone to win the bet.
I defined superhuman to mean that everyone will agree that it is more intelligent than any human being existing at that time.
I agree that the question of whether there is attention to Friendliness might be more problematic to determine. But “any attention to safety whatsoever” seems to me to be clearly stretching the idea of Friendliness—for example, someone could pay attention to safety by trying to make sure that the AI was mostly boxed, or whatever, and this wouldn’t satisfy Eliezer’s idea of Friendliness.
Right. And if this scenario happened, there would be a good chance that it would not be able to foom, or at least not within a week. Eliezer’s opinion seems to be that this scenario is extremely unlikely, in other words that the first AI will already be far more intelligent than the human race, and that even if it is running on an immense amount of hardware, it will have no need to acquire more hardware, because it will be able to construct nanotechnology capable of controlling the planet through actions originating on the internet as you suggest. And as you can see, he is very confident that all this will happen within a very short period of time.
Any specific scenario is going to have burdensome details, but that’s what you get if you ask for specific scenarios rather than general pressures, unless one spends a lot of time going through detailed possibilities and vulnerabilities. With respect to the specific example, regular human criminals routinely swindle or earn money anonymously online, and hack into and control millions of computers in botnets. Cloud computing resources can be rented with ill-gotten money.
In the unlikely event of a powerful human-indifferent AI appearing in the present day, a smartphone held by a human could provide sensors and communication to use humans for manipulators (as computer programs direct the movements of some warehouse workers today). Humans can be paid, blackmailed, deceived (intelligence agencies regularly do these things) to perform some tasks. An AI that leverages initial capabilities could jury-rig a computer-controlled method of coercion [e.g. a cheap robot arm holding a gun, a tampered-with electronic drug-dispensing implant, etc]. And as time goes by and the cumulative probability of advanced AI becomes larger, increasing quantities of robotic vehicles and devices will be available.
Thanks, yes I know about those arguments. One of the reasons I’m actually donating and accept AI to be one existential risk. I’m inquiring about further supporting documents and transparency. More on that here, especially check the particle collider analogy.
With respect to transparency, I agree about a lack of concise, exhaustive, accessible treatments. Reading some of the linked comments about marginal evidence from hypotheses I’m not quite sure what you mean, beyond remembering and multiplying by the probability that particular premises are false. Consider Hanson’s “Economic Growth Given Machine Intelligence”. One might support it with generalizations from past population growth in plants and animals, from data on capital investment and past market behavior and automation, but what would you say would license drawing probabilistic inferences using it?
Note that such methods might not result in the destruction of the world within a week (the guaranteed result of a superhuman non-Friendly AI according to Eliezer.)
What guarantee?.
With a guarantee backed by $1000.
The linked bet doesn’t reference “a week,” and the “week” reference in the main linked post is about going from infrahuman to superhuman, not using that intelligence to destroy humanity.
That bet seems underspecified. Does attention to “Friendliness” mean any attention to safety whatsoever, or designing an AI with a utility function such that it’s trustworthy regardless of power levels? Is “superhuman” defined relative to the then-current level of human (or upload, or trustworthy less intelligent AI) capacity with any enhancements (or upload speedups, etc)? What level of ability counts as superhuman? You two should publicly clarify the terms.
A few comments later on the same comment thread someone asked me how much time was necessary, and I said I thought a week was enough, based on Eliezer’s previous statements, and he never contradicted this, so it seems to me that he accepted it by default, since some time limit will be necessary in order for someone to win the bet.
I defined superhuman to mean that everyone will agree that it is more intelligent than any human being existing at that time.
I agree that the question of whether there is attention to Friendliness might be more problematic to determine. But “any attention to safety whatsoever” seems to me to be clearly stretching the idea of Friendliness—for example, someone could pay attention to safety by trying to make sure that the AI was mostly boxed, or whatever, and this wouldn’t satisfy Eliezer’s idea of Friendliness.
Ah. So an AI could, e.g. be only slightly superhuman and require immense quantities of hardware to generate that performance in realtime.
Right. And if this scenario happened, there would be a good chance that it would not be able to foom, or at least not within a week. Eliezer’s opinion seems to be that this scenario is extremely unlikely, in other words that the first AI will already be far more intelligent than the human race, and that even if it is running on an immense amount of hardware, it will have no need to acquire more hardware, because it will be able to construct nanotechnology capable of controlling the planet through actions originating on the internet as you suggest. And as you can see, he is very confident that all this will happen within a very short period of time.