Thanks. My thought is that any sufficiently intelligent AI would be capable of defeating any effort to prevent it from wireheading, and would resort to wireheading by default. It would know that humans don’t want it to wirehead so perhaps it might perceive humanity as a threat, however, it might realize humans aren’t capable of preventing it from wireheading and let humans live. In either event, it would just sit there doing nothing 24⁄7 and be totally satisfied in doing so. In other words, orthogonality wouldn’t apply to an intelligence capable of wireheading because wireheading would be its only goal. Is there a reason why an artifical super-intelligence would abstain from wireheading?
This isn’t the link I was thinking of (I was remembering something in the alignment discussion in the early days of lw, but I can’t find it), but this is probably a more direct answer to your request anyway: https://www.lesswrong.com/posts/FgsoWSACQfyyaB5s7/shutdown-seeking-ai
This is standard fare in the existing alignment discussion. See for instance https://www.lesswrong.com/posts/TtYuY2QBug3dn2wuo/the-problem-with-aixi or anything referring to wireheading.
Thanks. My thought is that any sufficiently intelligent AI would be capable of defeating any effort to prevent it from wireheading, and would resort to wireheading by default. It would know that humans don’t want it to wirehead so perhaps it might perceive humanity as a threat, however, it might realize humans aren’t capable of preventing it from wireheading and let humans live. In either event, it would just sit there doing nothing 24⁄7 and be totally satisfied in doing so. In other words, orthogonality wouldn’t apply to an intelligence capable of wireheading because wireheading would be its only goal. Is there a reason why an artifical super-intelligence would abstain from wireheading?