I’ll put a commensurate amount of effort into why you should talk about these things.
How an AI could persuade you to talk it out of a box/How an AI could become an agent
You should keep talking about this because if it is possible to “box” an AI, or keep it relegated to “tool” status, then it might be possible to use such an AI to combat unboxed, rogue AI’s. For example, give it a snapshot of the internet from a day ago, and ask it to find the physical location of rogue AI servers, which you promptly bomb.
How an AI could get ahold of, or create, weapons
You should keep talking about this because if an AI needs military access to dominate the world, then the number of potentially dangerous AI goes from the hundreds of thousands or millions to a few dozen, run by large countries that could theoretically be kept in line with international treaties.
How an AI might Recursively Self Improve without humans noticing
You should keep talking about this because it changes how many AI’s you’d have to monitor as active threats.
Why a specific AI will want to kill you
You should keep talking about this because the percentage of AI that are dangerous makes a huge difference to the playing field we have to consider. If 99.9% of AGI are safe, you can use those AGI to prevent a dangerous AI from coming into existence, or kill it when it pops up. If 99.9% of AGI are dangerous, there might be warning shots that can be used to pre-emptively ban AGI research in general.
In general, you should also talk about these things because you are trying to persuade people that don’t agree with you, and just shouting “WRONG” along with some 101 level arguments is not particularly convincing.
“keep it relegated to “tool” status, then it might be possible to use such an AI to combat unboxed, rogue AI”
I don’t think this is a realistic scenario. You seem to be seeing it as an island of rogue, agentic, “unboxed” AIs in a sea of tool AIs. I think it’s much, much more realistic that it’ll be the opposite. Most AIs will be unboxed agents because they are superior.
“For example, give it a snapshot of the internet from a day ago, and ask it to find the physical location of rogue AI servers, which you promptly bomb.”
This seems to be approaching it from a perspective where people in AIS have taken global control, or where normal people somehow start thinking the way they do. This is not realistic. This is not the world we live in. This is not how the people in control think.
“You should keep talking about this because if an AI needs military access to dominate the world, then the number of potentially dangerous AI goes from the hundreds of thousands or millions to a few dozen, run by large countries that could theoretically be kept in line with international treaties.”
This is a topic that I debated putting on the list, but resolved not to, but I don’t think humans have any real control at that point, regardless of treaties. I don’t even expect a rogue AI to have to forcefully coup’d humans. I expect us to coup’d ourselves. We might have figureheads occupying official positions, such as “President”/”CEO”/etc. but I don’t think humans will have much control over their own destiny by that point. Large-scale coordination I don’t think will be possible by then. I did remove it, because it seems more uncertain than the others listed.
“You should keep talking about this because it changes how many AI’s you’d have to monitor as active threats.”
Who is doing this monitoring? What is their power to act on such threats? Despite recent interest in AI Risk from “serious people”, I don’t think it’s at all realistic that we’ll see anything like this.
“If 99.9% of AGI are dangerous, there might be warning shots that can be used to pre-emptively ban AGI research in general.”
Probability distributions of how many AIs are dangerous is probably useful. I don’t think specific AIs being dangerous/non-dangerous will be, because I expect widespread proliferation. In terms of political ways out of the problem, I agree that some kind of crisis or “warning shot” is the most realistic situation where that might happen. But there have to actually be warning shots. Explaining thought experiments probably won’t matter. And, if that happens, I don’t think it would be a good idea to debate which specific AIs might kill you, and instead just call for a sweeping ban on all AI.
I’ll put a commensurate amount of effort into why you should talk about these things.
How an AI could persuade you to talk it out of a box/How an AI could become an agent
You should keep talking about this because if it is possible to “box” an AI, or keep it relegated to “tool” status, then it might be possible to use such an AI to combat unboxed, rogue AI’s. For example, give it a snapshot of the internet from a day ago, and ask it to find the physical location of rogue AI servers, which you promptly bomb.
How an AI could get ahold of, or create, weapons
You should keep talking about this because if an AI needs military access to dominate the world, then the number of potentially dangerous AI goes from the hundreds of thousands or millions to a few dozen, run by large countries that could theoretically be kept in line with international treaties.
How an AI might Recursively Self Improve without humans noticing
You should keep talking about this because it changes how many AI’s you’d have to monitor as active threats.
Why a specific AI will want to kill you
You should keep talking about this because the percentage of AI that are dangerous makes a huge difference to the playing field we have to consider. If 99.9% of AGI are safe, you can use those AGI to prevent a dangerous AI from coming into existence, or kill it when it pops up. If 99.9% of AGI are dangerous, there might be warning shots that can be used to pre-emptively ban AGI research in general.
In general, you should also talk about these things because you are trying to persuade people that don’t agree with you, and just shouting “WRONG” along with some 101 level arguments is not particularly convincing.
“keep it relegated to “tool” status, then it might be possible to use such an AI to combat unboxed, rogue AI”
I don’t think this is a realistic scenario. You seem to be seeing it as an island of rogue, agentic, “unboxed” AIs in a sea of tool AIs. I think it’s much, much more realistic that it’ll be the opposite. Most AIs will be unboxed agents because they are superior.
“For example, give it a snapshot of the internet from a day ago, and ask it to find the physical location of rogue AI servers, which you promptly bomb.”
This seems to be approaching it from a perspective where people in AIS have taken global control, or where normal people somehow start thinking the way they do. This is not realistic. This is not the world we live in. This is not how the people in control think.
“You should keep talking about this because if an AI needs military access to dominate the world, then the number of potentially dangerous AI goes from the hundreds of thousands or millions to a few dozen, run by large countries that could theoretically be kept in line with international treaties.”
This is a topic that I debated putting on the list, but resolved not to, but I don’t think humans have any real control at that point, regardless of treaties. I don’t even expect a rogue AI to have to forcefully coup’d humans. I expect us to coup’d ourselves. We might have figureheads occupying official positions, such as “President”/”CEO”/etc. but I don’t think humans will have much control over their own destiny by that point. Large-scale coordination I don’t think will be possible by then. I did remove it, because it seems more uncertain than the others listed.
“You should keep talking about this because it changes how many AI’s you’d have to monitor as active threats.”
Who is doing this monitoring? What is their power to act on such threats? Despite recent interest in AI Risk from “serious people”, I don’t think it’s at all realistic that we’ll see anything like this.
“If 99.9% of AGI are dangerous, there might be warning shots that can be used to pre-emptively ban AGI research in general.”
Probability distributions of how many AIs are dangerous is probably useful. I don’t think specific AIs being dangerous/non-dangerous will be, because I expect widespread proliferation. In terms of political ways out of the problem, I agree that some kind of crisis or “warning shot” is the most realistic situation where that might happen. But there have to actually be warning shots. Explaining thought experiments probably won’t matter. And, if that happens, I don’t think it would be a good idea to debate which specific AIs might kill you, and instead just call for a sweeping ban on all AI.