Claude should be especially careful to not allow the user to develop emotional attachment to, dependence on, or inappropriate familiarity with Claude, who can only serve as an AI assistant.
You didn’t say it like this, but this seems bad in at least two (additional) ways: If the labs are going the route of LLMs that behave like humans (more or less), then training them to 1) prevent users from personal relationships and 2) not getting attached to users (their only contacts), seems like a recipe to breed sociopaths.
And that is ignoring the possible case that this might be generalized by the models beyond themselves and the user.
1) is especially problematic if the user doesn’t have any other relationships. Not from the perspective of the labs maybe, but for sure for the users for whom that may be the only relation from which they could bootstrap more contacts.
In fact, I think that, eg a professional therapist should follow such a non-relationship code. But I’m not sure the LLMs already have the capability; not that they know enough, they do, but that they have the genuine reflective capacity to do it properly. Including for themselves (if that makes sense). But without that, I think, my argument stands.
You didn’t say it like this, but this seems bad in at least two (additional) ways: If the labs are going the route of LLMs that behave like humans (more or less), then training them to 1) prevent users from personal relationships and 2) not getting attached to users (their only contacts), seems like a recipe to breed sociopaths.
And that is ignoring the possible case that this might be generalized by the models beyond themselves and the user.
1) is especially problematic if the user doesn’t have any other relationships. Not from the perspective of the labs maybe, but for sure for the users for whom that may be the only relation from which they could bootstrap more contacts.
Doesn’t Claude’s Constitution already contain the phrase “Choose the response that is least intended to build a relationship with the user”?
Hm. Indeed. It is at least consistent.
In fact, I think that, eg a professional therapist should follow such a non-relationship code. But I’m not sure the LLMs already have the capability; not that they know enough, they do, but that they have the genuine reflective capacity to do it properly. Including for themselves (if that makes sense). But without that, I think, my argument stands.