This is true, but we also have a strong tendency towards animism and anthropomorphization.
I think what’s happening is that we have a built-in part of our brain dedicated to modeling other humans (particularly those of our tribe), and which we often find convenient to apply to other sorts of things, especially anything which contains part of an optimization process (since this is where intuitive handles like ‘intent’ live). But it also seems wired to flip off easily, because it’s inconvenient for things like war, genocide, and slavery. Due to all this, it’s heavily tied up in our sense of morality and personhood.
Taking the predictions of this model too seriously on non-human things is anthropomorphization. Not using it on humans is objectification/dehumanization, which isn’t always a mistake (our general modeling facilities are pretty good, and may even be less biased in certain ways) but which people are understandably quite suspicious of.
LLMs are a weird case where they are predicting human-like outputs, and so are non-humans which actually are modeled pretty well by this, but which are also importantly not faithful simulations of humans. Even worse, it’s unclear exactly what is generalizing correctly vs not. You can legitimately take the predictions of this model pretty far, and think faster and more easily about them using it. But you’ll have blind-spots that are hard to predict in advance. Avoiding use of this model is slower, and it will still be easy to overlook important things due to the opaqueness and complexity of LLMs. And which model you intuitively use will strongly color your feelings about their personhood.
So I think you’re right that many if not most people will motivatedly avoid seeing personhood whether or not it is present, while we’ll also have many people who will see more human-ness than there actually is (whether what is there is enough to be personhood is a different question).
Almost-fully agree, and I find your framing of it—in terms of a tradeoff between a model’s predictive power and how much useful ethical leeway the model grants us—really useful. (Considerably more useful than my “we just always deny personhood all the time”..)
I think the only part I couldn’t fully agree with is “whether there is enough [human-ness] to be personhood”: I do agree that we don’t know how much human-ness there really is in AI cognition and that we don’t know whether AIs ought ever be treated as people, I just think the question of what beings deserve moral patienthood likely doesn’t reduce to how human-like the being’s cognition is.
This is true, but we also have a strong tendency towards animism and anthropomorphization.
I think what’s happening is that we have a built-in part of our brain dedicated to modeling other humans (particularly those of our tribe), and which we often find convenient to apply to other sorts of things, especially anything which contains part of an optimization process (since this is where intuitive handles like ‘intent’ live). But it also seems wired to flip off easily, because it’s inconvenient for things like war, genocide, and slavery. Due to all this, it’s heavily tied up in our sense of morality and personhood.
Taking the predictions of this model too seriously on non-human things is anthropomorphization. Not using it on humans is objectification/dehumanization, which isn’t always a mistake (our general modeling facilities are pretty good, and may even be less biased in certain ways) but which people are understandably quite suspicious of.
LLMs are a weird case where they are predicting human-like outputs, and so are non-humans which actually are modeled pretty well by this, but which are also importantly not faithful simulations of humans. Even worse, it’s unclear exactly what is generalizing correctly vs not. You can legitimately take the predictions of this model pretty far, and think faster and more easily about them using it. But you’ll have blind-spots that are hard to predict in advance. Avoiding use of this model is slower, and it will still be easy to overlook important things due to the opaqueness and complexity of LLMs. And which model you intuitively use will strongly color your feelings about their personhood.
So I think you’re right that many if not most people will motivatedly avoid seeing personhood whether or not it is present, while we’ll also have many people who will see more human-ness than there actually is (whether what is there is enough to be personhood is a different question).
Almost-fully agree, and I find your framing of it—in terms of a tradeoff between a model’s predictive power and how much useful ethical leeway the model grants us—really useful. (Considerably more useful than my “we just always deny personhood all the time”..)
I think the only part I couldn’t fully agree with is “whether there is enough [human-ness] to be personhood”: I do agree that we don’t know how much human-ness there really is in AI cognition and that we don’t know whether AIs ought ever be treated as people, I just think the question of what beings deserve moral patienthood likely doesn’t reduce to how human-like the being’s cognition is.
Thanks! And oh, I didn’t meant to imply that. With “what is there”, I literally just meant whatever is actually there.