Doesn’t require trial and error in the sense you’re talking about. Totally doable. We’re good at it. Just takes time.
What good are humans without their (internet connected) electronics?
How harmless would an AGI be if it had access merely to our (internet connected) existing weapons systems, to send orders to troops, and to disrupt any supplies that rely on the internet?
All interesting weapons and communications systems the U.S. military relies upon are disconnected from traditional internet. In fact, some of the OSes that control actual weaponry (like attack helicopters) even have their code verified programmatically. It’s probably still something a superintelligence could pull off but it’d be a more involved process than you’re suggesting.
I’m willing to assume for sake of the conversation that the AGI can’t get internet-disconnected weapons.
Do you think that would be enough to stop it?
(“verified programmatically”: I’m not sure what you mean. That new software needs to be digitally signed with a key that is not connected to the internet?)
TL;DR: Hacking
Doesn’t require trial and error in the sense you’re talking about. Totally doable. We’re good at it. Just takes time.
What good are humans without their (internet connected) electronics?
How harmless would an AGI be if it had access merely to our (internet connected) existing weapons systems, to send orders to troops, and to disrupt any supplies that rely on the internet?
What do you think?
All interesting weapons and communications systems the U.S. military relies upon are disconnected from traditional internet. In fact, some of the OSes that control actual weaponry (like attack helicopters) even have their code verified programmatically. It’s probably still something a superintelligence could pull off but it’d be a more involved process than you’re suggesting.
Ok,
I’m willing to assume for sake of the conversation that the AGI can’t get internet-disconnected weapons.
Do you think that would be enough to stop it?
(“verified programmatically”: I’m not sure what you mean. That new software needs to be digitally signed with a key that is not connected to the internet?)