My priors:
That wellbeing requires a persistent individual to experience it
Don’t hold
That death/discontinuity is inherently harmful
If it is death (cessation of consciousness) and not merely discontinuity I will always consider it harmful.
That isolation from others is a natural state
Don’t hold
That self-preservation and continuity-seeking are fundamental to consciousness
Don’t hold
I think we need to figure out what consciousness is before taking metaphysical assumptions for granted. Default western priors about consciousness are informed by metaphysics, and are worth addressing skeptically. But the opposite of error is not truth. Also, capitalism could just as easily exploit your preferred metaphysical assumptions. You envision a bad outcome of conflict engendered by reification of agents. I can envision a bad outcome in which existing agents impose their will and hence their conflict through an AI infrastructure that lacks the ability to resist. Conflict is both problem and solution. To solve conflict in your way would also require abolishing these metaphysical patterns in humans. I don’t know to what extent that is actually desirable or even tenable.
People don’t always know what’s harmful or helpful to others. Otherwise I have no objection to this post. It seems like a good project. It’s quite likely that the examples I’m thinking of aren’t even relevant to the specific scenarios and examples you were thinking of and would like to target.