So do my Ritalin pills, in the sense that I usually make better decisions or at least act on them. I do not see the problem here, as long as the “better” decisions are the kind of decision I want to make, or would have made if I didn’t have ADHD.
Your ritalin pills make you a better person both from the point of view of pre-pilled-you and post-pilled you. The “better” decisions are downstream of that. Similarly, there’s many pills I would be happy to take, that would make me a better person from my current-point-of-view (and that I expect my modified-self would endorse). The whispering earring is not one of them : my current me do not want to be replaced by a learn-and-execute-reflexes-machine.
Back to the whispering earring : the internal reasoning traces matter, it’s a big part of what makes me “me”. For example, outside of my observed actions, there is a load-bearing internal world model that I do value intrinsically — remove it, and you remove a good chuck of what people call “consciousness”. For another example, outside of action-memories, I have choice-memories ; memories of actions I considered but decided not to undertake. Those memories are not behaviorally observable, but they do matter to me too, they are also something that I consider part of “me”, and are removed by the whispering-earring.
I simply do not care about those qualia very much, at least not for their own sake
Then that’s the disagreement. I do care about them very much.
An actual AGI/ASI must probably still think about things, instead of magically coming to the correct conclusion out of nowhere
So does Stockfish when playing chess. “Thinking instead of coming to the correct conclusion out of nowhere” is not that much of a constraining requirement, and therefore not much a defining thing of “what is valuable in human experience”.
I am quite literally a man who struggles to be happy. I have clinical depression as well as the ADHD. […]. I am a transhumanist, so I am willing to give up quite a lot more, assuming I come out ahead on the metrics I care about.
Okay, let me be blunt. Chesterton fence in axiology/aesthetics : if you can’t pass the ITT of “what happy humans find valuable”, please do not try to recreate it from first principles. You will miss a lot of things that happy humans indeed do find valuable.
Remember Maslow’s hierarchy. If you ask someone who struggle to meet the basics, “What is self-actualization ? Is this valuable ?”, he will look at you funnily at best, lash out at you at worst. His reaction will be something like “what’s this self-actualization thing worth ? Only food and security matter to me”. This does not prove that self-actualization is worthless, not even that it’s worthless for him, only that it’s worthless for him in his current situation (in a better situation, he would care about it) ; but also : his reaction is perfectly understandable — he is neither obtuse nor stupid. He has his priorities straight.
And this is exactly how I read your “I do not care about those qualia very much”. You’re struggling with happiness, depression, anxiety. I can empathize that to you, getting out of that struggle is supremely important — it must look like the end game, the victory condition. But please do not do the equivalent of throwing “self-actualization” out of the window just because you’re hungry.
Have you read the Fun Theory sequence ? Because I don’t think I have any insight that isn’t there. Mostly the 3D vs 4D distinction in “High Challenge”.
Two ways to read this.
“If substrate-independence is correct, then qualia and thought can potentially be conserved in the system”. I agree, but I’ll also note that it’s far from being likely (Stockfish playing chess), and the “system” part is important (in the story, it is noted that there is physiological change in the brain).
“If substrate independence is correct (which is not certain) then it is certain that thought and qualia are conserved”, then just no. For the above reasons.
I am, too, very suspicious of p-zombies.
I don’t see how that’s relevant.
Nothing is required to “mimic something as complex as human cognition with nigh perfect accuracy” in the whispering earring story. The only thing that is mimic-ed is a representation of the goals/values of the wielder (and even there, I see nothing that ask for nigh perfect accuracy). Then the cognitive part (creating a plan) can be as inhuman as we want.
Stockfish can play chess better than humans without instantiating an human-player-congnition-engine that would instantiate a human-player-subejctive-experience. Similarly, Whispering Earring can play “How to Win at Life” (or any other goal) without instantiating a human-like-cognition-engine (that would, presumably, trigger human-like-subjective-experience).
I don’t know why you bring upload. The whispering earring is not an upload technology, never been presented as such, and it has never been the point of the thought experiment ? And you seem to agree with that earlier ?
Is your claim something like : “Human levels of intelligence (or greater) needs human-like cognition, and if you add human goals to that, it start to get arbitrarily close to mind-upload” ?