Welcome and thanks for engaging too! Yeah, I think this is still very subjective and open-ended and we can’t accurately predict if Superintelligent AI would even want to care about the existence of humans or not. The statement “superintelligent AI will have sufficient means to ensure its survival irrespective of human existence” is correct I think, but remember that AI alignment research of all AI companies run by humans would all be trying to control Superintelligent AI in the making.
So I don’t think it would be like- “I woke up today and I found that we made Superintelligent AI in the lab today! Will we be able to control it now?”
It would be more like- “This AI is getting more capable and intelligent day by day. Am I sure I am able to contain it securely?”
In short, a sort of continuous trajectory towards AGI or superintelligence.
Now I think there is 2 separate things that point #3 and point 1 in my comment are talking about implicitly. #3 IMO is the scenario when AI is able to escape from the servers of frontier AI labs, has successfully replicated itself on other distributed systems and then humans trying to somehow contain it and shut it down. Whereas point 1 in the comment is predominantly assuming the scenario when the AI getting intelligent day by day is still on the servers of the frontier AI labs. Thus, there isn’t any contradiction here.
Welcome and thanks for engaging too! Yeah, I think this is still very subjective and open-ended and we can’t accurately predict if Superintelligent AI would even want to care about the existence of humans or not. The statement “superintelligent AI will have sufficient means to ensure its survival irrespective of human existence” is correct I think, but remember that AI alignment research of all AI companies run by humans would all be trying to control Superintelligent AI in the making.
So I don’t think it would be like- “I woke up today and I found that we made Superintelligent AI in the lab today! Will we be able to control it now?”
It would be more like- “This AI is getting more capable and intelligent day by day. Am I sure I am able to contain it securely?”
In short, a sort of continuous trajectory towards AGI or superintelligence.
Now I think there is 2 separate things that point #3 and point 1 in my comment are talking about implicitly. #3 IMO is the scenario when AI is able to escape from the servers of frontier AI labs, has successfully replicated itself on other distributed systems and then humans trying to somehow contain it and shut it down. Whereas point 1 in the comment is predominantly assuming the scenario when the AI getting intelligent day by day is still on the servers of the frontier AI labs. Thus, there isn’t any contradiction here.