I’m sure my mom would have found this more convincing, but it struggles from the fact that an obvious counter-argument exists. Instead of not building AGI at all, people can just say “Well, just don’t put AGI in control of the military then!”
That counterargument is unfortunately always available for all scenarios, including non-AI cases. “Just don’t do the bad thing.” I’m not sure what you think specifically in this scenario triggers it to be more salient. Is it “The Military” as a common adversary? If I think about a scenario where AI is used to optimize or “control” the energy grid of supply chain logistics, would that be different?
This specific scenario involves us making a second obvious mistake (putting the AI in charge of the military) vs. scenarios that just involve us creating AI, and then the AI does things of its own volition like escaping and creating bioweapons that are dangerous without us having to do anything at all.
OK. That seems to require AI hacking out of a box, which is unbelievable as per rule 4 or 8. Or do more mundane cases like AI doing economic transactions or research count?
I’m sure my mom would have found this more convincing, but it struggles from the fact that an obvious counter-argument exists. Instead of not building AGI at all, people can just say “Well, just don’t put AGI in control of the military then!”
That counterargument is unfortunately always available for all scenarios, including non-AI cases. “Just don’t do the bad thing.” I’m not sure what you think specifically in this scenario triggers it to be more salient. Is it “The Military” as a common adversary? If I think about a scenario where AI is used to optimize or “control” the energy grid of supply chain logistics, would that be different?
This specific scenario involves us making a second obvious mistake (putting the AI in charge of the military) vs. scenarios that just involve us creating AI, and then the AI does things of its own volition like escaping and creating bioweapons that are dangerous without us having to do anything at all.
OK. That seems to require AI hacking out of a box, which is unbelievable as per rule 4 or 8. Or do more mundane cases like AI doing economic transactions or research count?