Yeah, I don’t think any of that holds water. There are two separate questions, to which my answers are “maybe” and “no”.
The first question is whether whatever is inside the earring is a person. To this my answer is “maybe, most likely no”. In fact I think that simulation of conscious behavior doesn’t even necessarily involve consciousness, and if optimization pressure is involved (e.g. if the earring tries to simplify its internal computation to save resources), then there’s a high risk of ending up with something that dutifully says “I’m conscious” without actually being so. For more detail, I wrote out this argument before w.r.t. consciousness of LLMs and I think it applies here too.
The second question is whether, conditional on the earring-software being a person, that person is you. To this my answer is a simple “no”. The behavior of that person is very much not identical to yours, so it’s much worse than the jewelhead situation. And it might even differ from field to field. For example, if someone is a complete fuckup in all areas of life but a genius musician, the earring-version of him might be very similar to him in matters of music, but very different from him in all other matters. Or consider this: the earring upload of an all-around competent person will be very similar to them, but the earring-upload of an all-around fuckup will be very different. We can’t in good conscience say that these are the same fidelity. Therefore we have to consider fidelity, and the whole argument falls apart.
I am agnostic on whether LLMs have qualia. I am less agnostic on whether the earring+human system has qualia, but not by a massive margin. That’s why I floated my points as being suggestive and at the very least internally consistent, rather than authoritative settled.
>The second question is whether, conditional on the earring-software being a person, that person is you. To this my answer is a simple “no”. The behavior of that person is very much not identical to yours, so it’s much worse than the jewelhead situation. And it might even differ from field to field. For example, if someone is a complete fuckup in all areas of life but a genius musician, the earring-version of him might be very similar to him in matters of music, but very different from him in all other matters. Or consider this: the earring upload of an all-around competent person will be very similar to them, but the earring-upload of an all-around fuckup will be very different. We can’t in good conscience say that these are the same fidelity. Therefore we have to consider fidelity, and the whole argument falls apart.
I will re-employ the stimulant analogy, since I know very well from the inside what it feels like.
Without stimulants, my ADHD makes me worse, in most ways I care about. It makes me lazy, prone to procrastination, to give insufficient weight to my academic priorities. I don’t think I was a complete fuck-up before I was diagnosed and treated, but they have let me achieve very many things I would not achieve without their assistance.
Barring minor to moderate discomfort and inconvenience that I willingly accept, this is a ridiculously positive trade. I am suitably grateful.
Behaviorally? Less time lying in bed. More time studying. Better focus and less prone to error. If you squint, that is a different person too, compared to who I was before or after the meds wear off. But I couldn’t care less, I want those improvements.
The text of the story, in no uncertain terms, states that the earring always follows the user’s desires or at least warns them when its advice might not align with them. To quote:
>It is not a taskmaster, telling you what to do in order to achieve some foreign goal. It always tells you what will make you happiest. If it would make you happiest to succeed at your work, it will tell you how best to complete it. If it would make you happiest to do a half-assed job at your work and then go home and spend the rest of the day in bed having vague sexual fantasies, the earring will tell you to do that. The earring is never wrong
Emphasis added. In other words, the earring never does anything that a person wouldn’t endorse (at least without some kind of warning). Obviously someone who is deeply lazy, akratic or “fucked up” will benefit more than someone who already has their life together. I do not see an issue with this, anymore than I see an issue with the fact that people with ADHD get a proportionally larger boost from the drugs. The fuck-up wants to be better. So do I. And this isn’t some twisted, malign and superficial form of happiness either, the earring isn’t wireheading them or telling them to resort to addictive substances by default.
Yeah, I don’t think any of that holds water. There are two separate questions, to which my answers are “maybe” and “no”.
The first question is whether whatever is inside the earring is a person. To this my answer is “maybe, most likely no”. In fact I think that simulation of conscious behavior doesn’t even necessarily involve consciousness, and if optimization pressure is involved (e.g. if the earring tries to simplify its internal computation to save resources), then there’s a high risk of ending up with something that dutifully says “I’m conscious” without actually being so. For more detail, I wrote out this argument before w.r.t. consciousness of LLMs and I think it applies here too.
The second question is whether, conditional on the earring-software being a person, that person is you. To this my answer is a simple “no”. The behavior of that person is very much not identical to yours, so it’s much worse than the jewelhead situation. And it might even differ from field to field. For example, if someone is a complete fuckup in all areas of life but a genius musician, the earring-version of him might be very similar to him in matters of music, but very different from him in all other matters. Or consider this: the earring upload of an all-around competent person will be very similar to them, but the earring-upload of an all-around fuckup will be very different. We can’t in good conscience say that these are the same fidelity. Therefore we have to consider fidelity, and the whole argument falls apart.
I am agnostic on whether LLMs have qualia. I am less agnostic on whether the earring+human system has qualia, but not by a massive margin. That’s why I floated my points as being suggestive and at the very least internally consistent, rather than authoritative settled.
>The second question is whether, conditional on the earring-software being a person, that person is you. To this my answer is a simple “no”. The behavior of that person is very much not identical to yours, so it’s much worse than the jewelhead situation. And it might even differ from field to field. For example, if someone is a complete fuckup in all areas of life but a genius musician, the earring-version of him might be very similar to him in matters of music, but very different from him in all other matters. Or consider this: the earring upload of an all-around competent person will be very similar to them, but the earring-upload of an all-around fuckup will be very different. We can’t in good conscience say that these are the same fidelity. Therefore we have to consider fidelity, and the whole argument falls apart.
I will re-employ the stimulant analogy, since I know very well from the inside what it feels like.
Without stimulants, my ADHD makes me worse, in most ways I care about. It makes me lazy, prone to procrastination, to give insufficient weight to my academic priorities. I don’t think I was a complete fuck-up before I was diagnosed and treated, but they have let me achieve very many things I would not achieve without their assistance.
Barring minor to moderate discomfort and inconvenience that I willingly accept, this is a ridiculously positive trade. I am suitably grateful.
Behaviorally? Less time lying in bed. More time studying. Better focus and less prone to error. If you squint, that is a different person too, compared to who I was before or after the meds wear off. But I couldn’t care less, I want those improvements.
The text of the story, in no uncertain terms, states that the earring always follows the user’s desires or at least warns them when its advice might not align with them. To quote:
>It is not a taskmaster, telling you what to do in order to achieve some foreign goal. It always tells you what will make you happiest. If it would make you happiest to succeed at your work, it will tell you how best to complete it. If it would make you happiest to do a half-assed job at your work and then go home and spend the rest of the day in bed having vague sexual fantasies, the earring will tell you to do that. The earring is never wrong
Emphasis added. In other words, the earring never does anything that a person wouldn’t endorse (at least without some kind of warning). Obviously someone who is deeply lazy, akratic or “fucked up” will benefit more than someone who already has their life together. I do not see an issue with this, anymore than I see an issue with the fact that people with ADHD get a proportionally larger boost from the drugs. The fuck-up wants to be better. So do I. And this isn’t some twisted, malign and superficial form of happiness either, the earring isn’t wireheading them or telling them to resort to addictive substances by default.