1- I really think it’s much simpler than that. Just look at the cold war, look at how one person with a past story of frustration, etc. (which by the way is usually totally accesible for an AGI) could end up selling his own people just out of some unattended old feeling. We, humans, are very easy to manipulate. For me our weakest point is that nobody really has the complete picture of whats going on, and I mean NOBODY. So the chances of an AGI being shut down are small, it would replicate itself, morph, we’re not even good at fighting ransomware which is commanded by people. The connected network of computers, and the huge clouds we’ve created are somewhat inhospit places for us, it’s like being blind in the middle of a forest. The AGI would move much better than us. Our only chance would lie in working all toghether as a team, but there’s always going to be some group of people that would end up aligning with the AGI, history has proven time and time again that joining all humans on a common task has a really small (almost inexistent) chance of happening. My 0.02.
2- Again I don’t feel there’s much knowledge to be gained in my scenario, it’s just about controlling us. An AGI could perfectly well speak, generate (fake) video, communicate over our preferred channels of communication, etc. I don’t see much information, much trial and error or anything needed. All the tools an AGI would need to (again) dominate us are already here.
Hi all, I’m really sorry I’ve not yet been able to read the whole list of comments and replies, but I’d like to rise the point that usually an intelligence which is one order of magnitude or more than the existing ones can controll them at will. We humans are able to control dogs and make them kill each other (dog fights) beacuse we kind of understand the way they react to different stimulus. I don’t see why the AGI would need to spend so much time preparing robots, it could just keep an army of humans the size it will and this army could perfectly well do anything it needs given that the AGI is far superior to us regarding intelligence. Also humans would probably never know that they’re being commanded by an AGI, I don’t feel it’s too hard to convince a human to kill another human for a high porpouse. What I mean is that I think the whole point of analyzing the robots, etc is useless, what should be analyzed is how long would it take an AGI to make humans believe they’re fighting for a higher porpouse (as in the cruzades for example) and have an army of humans do whatever it takes. Of course that’s not hte end of humans, but at least it’s the end of “free” humans (if that’s something we are right now, which is also a matter of discussion...)
Sorry for my english, not my native tongue.
(minor corrections, sorry again)