The biggest issue with control is that if we assume superintelligence a priori then it would be able to make the best decisions to evade detection, to avoid being caught, to even appear stupid enough that humans would not be very worried. I think it would be impossible to guarantee any kind of control given we don’t really know what intelligence even is. It is not impossible to imagine that it already exists as a substrate of the communication/financial/bureaucratic network we have created.
I find most interesting that we ignore that even the dumbest of super intelligences would start from having a very clear understanding of all the content on this section.
Notice “a priori” usually means something else altogether.
What you mean is closer to “by definition”.
Finally, given your premise that we do not know what intelligence is (thus don’t know the Super version of it either) it’s unclear where this clear thread-understanding ability stems from.
Absolutely! It’s helpful to remember we are talking about an intelligence that is comparable to our own. (The great danger only arises with that proximity.) So if you would not feel comfortable with the AI listening in on this conversation (and yes it will do its research, including going back to find this page), you have not understood the problem. The only safety features that will be good enough are those designed with the full knowledge that the AI is sitting at the table with us, having heard every word. That requires a pretty clever answer and clever is where the AI excels!
Furthermore this will be the luxury problem, after humanity has cracked the nut of mutual agreement on our approach to AI. That’s the only way to avoid simply succumbing to infighting; meaning whomever’s first to give the AI what it wants “wins”, (perhaps by being last in line to be sacrificed).
The biggest issue with control is that if we assume superintelligence a priori then it would be able to make the best decisions to evade detection, to avoid being caught, to even appear stupid enough that humans would not be very worried. I think it would be impossible to guarantee any kind of control given we don’t really know what intelligence even is. It is not impossible to imagine that it already exists as a substrate of the communication/financial/bureaucratic network we have created.
I find most interesting that we ignore that even the dumbest of super intelligences would start from having a very clear understanding of all the content on this section.
Notice “a priori” usually means something else altogether.
What you mean is closer to “by definition”.
Finally, given your premise that we do not know what intelligence is (thus don’t know the Super version of it either) it’s unclear where this clear thread-understanding ability stems from.
Absolutely! It’s helpful to remember we are talking about an intelligence that is comparable to our own. (The great danger only arises with that proximity.) So if you would not feel comfortable with the AI listening in on this conversation (and yes it will do its research, including going back to find this page), you have not understood the problem. The only safety features that will be good enough are those designed with the full knowledge that the AI is sitting at the table with us, having heard every word. That requires a pretty clever answer and clever is where the AI excels!
Furthermore this will be the luxury problem, after humanity has cracked the nut of mutual agreement on our approach to AI. That’s the only way to avoid simply succumbing to infighting; meaning whomever’s first to give the AI what it wants “wins”, (perhaps by being last in line to be sacrificed).