To those who say they don’t want to be wireheaded, how do you really know that, when you haven’t tried wireheading?
Same reason I don’t try heroin. Wireheading (as generally conceived) imposes a predictable change on the user’s utility function; huge and irreversible. Gathering this information is not without cost.
I’m not suggesting that you try wireheading now, I’m saying that an FAI can obtain this information without a high cost, and when it does, it may turn out that you actually do prefer to be wireheaded.
Same reason I don’t try heroin. Wireheading (as generally conceived) imposes a predictable change on the user’s utility function; huge and irreversible. Gathering this information is not without cost.
I’m not suggesting that you try wireheading now, I’m saying that an FAI can obtain this information without a high cost, and when it does, it may turn out that you actually do prefer to be wireheaded.
That’s possible (especially the non-addictive type of wire heading).
Though this does touch upon issues of autonomy—I’d like the AI to run it by me, even though it will have correctly predicted that I’d accept.