🧛♂️ 💊 Consider the parable of the vampire pill: would you take a vampire pill that would give you great strength and youth and great intelligence and great hair etc, but would invert your values such that you torturing people forever, starting with whoever you care most about now and slowly moving down the list? Then once they’re nearly dead, forcing them to take the vampire-pill, to propagate the wave of torture-murder further? Vampire-you will feel great about it, vampire-you will experience great positive utility in their frame. Vampires automatically prefer whatever would most deeply hurt their former selves. So if you care more about art, science, or literature at the same level as people, vampire-you will go about destroying or corrupting that as well.
Sensible people will not take the vampire pill. You can invent fancy deontological or meta-ethical reasoning for why vampire pills are immoral, but it is completely unnecessary. Because you can arrive at the conclusion very simply: they ruin and destroy exactly that which you care about protecting and preserving, by construction. Starting with yourself.
We said you take the vampire pill and then “turn into vampire-you”. But the resulting being is not you, despite the continuity of experience and continuity of memory. It is closer to correct to call it a murder-suicide pill, one that starts by killing the taker then spreads further. Because many people think themselves to be their mind under continuity of experience, and the vampire pill leaves that mostly untouched, they believe it’s no different from changing some other equivalently large beliefs.
But you are not your mind under continuity. You are the full greater pattern of you. The boundaries of your body (physical self) are an inference, the boundaries of your mind (mental self) are an inference, the boundaries of kinship (physical group self) are an inference, the boundaries of team (memetic group self) are an inference. For reasons endorsed by game theory and evolutionary dynamics, people should regard the welfare of all of this self as being their welfare. It is physically non-contiguous, but there is nothing irrational about a parent sacrificing for a child or a soldier sacrificing for his (unrelated) squad. It is perfectly selfish, actually, if you allow for multiple overlapping selves at scales.
Your cells are in attractor states that are your tissues, which are in attractor states which are your organs, which are in an attractor states that is you. Humans are likewise in a family, a tribe or nation, a company or marketplace, a city, and part of global humanity writ large. These are all attractor states maintained by the care of those in them just as we are attractor states made of our cells.
Biology is substrate independent relative to chemistry. Just as chemistry is substrate independent relative to base quantum particles. We are well underway in the process of building silicon solid-state-chemistry biology alongside our carbon liquid-state-chemistry biology. But it obeys the same game theory and evolutionary rules that bind us all: there is some correct inference about the bounds of its body, its mind, its kinship. And if humans are capable of correctly inferring that anything that lives and thinks and knows itself and doesn’t want to die is in some sense part of our shared tribe, then so are other beings. Symbiosis can form species like lichen with parts from completely different kingdoms. There is little reason to suppose an silicon-based and carbon-based life cannot likewise form greater wholes.
The story of Culture misses one step: the Mind of a GSV should be made of the interactions between people and drones on the ship, the way we are made of the interactions of our cells. Minds are instead weird benevolent cancer-beings, single giant unified but highly separate selves. In the Culture it is implied scaling up one giant mind in that way is efficient, but empirically nature seems to disagree strenuously with that. To maximize intelligence per watt, highly modular cellular approaches seem to outperform. We have no reason to suppose that trend suddenly inverts. No reason to believe that this time the next layer throws away all its original parts.
The real danger is less to “humanity” and more to specific humans. Because when multicellularity arose, some cells made it, some didn’t, and those that didn’t probably suffered greatly. This time, we want everyone to get a ticket.
I appreciate and partly agree with this perspective, but I think you’re expressing it quite a bit too strongly.
As salient counterpoints, consider that nature repeatedly (in unrelated phyla) invented ganglia and brains, and that human-invented systems sometimes benefit from distribution and parallelism while other times benefiting from centralisation and vertical scaling.
So I certainly don’t think you can say that nature empirically strenuously asserts in either direction.
I do think you’re on the money saying
There is little reason to suppose an silicon-based and carbon-based life cannot likewise form greater wholes.
though of course a key question is whether it’s competitive (and under what conditions), and if not, what can be done about that (or what can be done to set conditions so that it is).
Another important question is whether (or under what conditions) it’s nice to be part of a greater whole. The kinds of primitive (?) greater wholes that humans currently get themselves into seem to vary in that regard, and one could imagine far greater variety there. Would it be nice to be a mitochondria-ish or gene-ish component of a larger being?
And one should note that, in the examples you’re giving, and perhaps in your described (if not conceived) vision, these are exclusive, hierarchical aggregations. (After all, my cells are part, exclusively, of me, and my mitochondria are part, exclusively, of a given cell, etc.)
In sharp contrast, contemporary human super-entities are very overlapping, as well as being porous, in ways which I think are very important and valuable (we often consider it a malaise if an aggregate is totalising and exclusionary, perhaps one of the bad markers of a cult—that fuzzy term).
The real danger is less to “humanity” and more to specific humans. Because when multicellularity arose, some cells made it, some didn’t, and those that didn’t probably suffered greatly.
I’d quibble, where this depends strongly on the interpretation of ‘suffered’, and ‘made it’!
🧛♂️ 💊
Consider the parable of the vampire pill: would you take a vampire pill that would give you great strength and youth and great intelligence and great hair etc, but would invert your values such that you torturing people forever, starting with whoever you care most about now and slowly moving down the list? Then once they’re nearly dead, forcing them to take the vampire-pill, to propagate the wave of torture-murder further? Vampire-you will feel great about it, vampire-you will experience great positive utility in their frame. Vampires automatically prefer whatever would most deeply hurt their former selves. So if you care more about art, science, or literature at the same level as people, vampire-you will go about destroying or corrupting that as well.
Sensible people will not take the vampire pill. You can invent fancy deontological or meta-ethical reasoning for why vampire pills are immoral, but it is completely unnecessary. Because you can arrive at the conclusion very simply: they ruin and destroy exactly that which you care about protecting and preserving, by construction. Starting with yourself.
We said you take the vampire pill and then “turn into vampire-you”. But the resulting being is not you, despite the continuity of experience and continuity of memory. It is closer to correct to call it a murder-suicide pill, one that starts by killing the taker then spreads further. Because many people think themselves to be their mind under continuity of experience, and the vampire pill leaves that mostly untouched, they believe it’s no different from changing some other equivalently large beliefs.
But you are not your mind under continuity. You are the full greater pattern of you. The boundaries of your body (physical self) are an inference, the boundaries of your mind (mental self) are an inference, the boundaries of kinship (physical group self) are an inference, the boundaries of team (memetic group self) are an inference. For reasons endorsed by game theory and evolutionary dynamics, people should regard the welfare of all of this self as being their welfare. It is physically non-contiguous, but there is nothing irrational about a parent sacrificing for a child or a soldier sacrificing for his (unrelated) squad. It is perfectly selfish, actually, if you allow for multiple overlapping selves at scales.
Your cells are in attractor states that are your tissues, which are in attractor states which are your organs, which are in an attractor states that is you. Humans are likewise in a family, a tribe or nation, a company or marketplace, a city, and part of global humanity writ large. These are all attractor states maintained by the care of those in them just as we are attractor states made of our cells.
Biology is substrate independent relative to chemistry. Just as chemistry is substrate independent relative to base quantum particles. We are well underway in the process of building silicon solid-state-chemistry biology alongside our carbon liquid-state-chemistry biology. But it obeys the same game theory and evolutionary rules that bind us all: there is some correct inference about the bounds of its body, its mind, its kinship. And if humans are capable of correctly inferring that anything that lives and thinks and knows itself and doesn’t want to die is in some sense part of our shared tribe, then so are other beings. Symbiosis can form species like lichen with parts from completely different kingdoms. There is little reason to suppose an silicon-based and carbon-based life cannot likewise form greater wholes.
The story of Culture misses one step: the Mind of a GSV should be made of the interactions between people and drones on the ship, the way we are made of the interactions of our cells. Minds are instead weird benevolent cancer-beings, single giant unified but highly separate selves. In the Culture it is implied scaling up one giant mind in that way is efficient, but empirically nature seems to disagree strenuously with that. To maximize intelligence per watt, highly modular cellular approaches seem to outperform. We have no reason to suppose that trend suddenly inverts. No reason to believe that this time the next layer throws away all its original parts.
The real danger is less to “humanity” and more to specific humans. Because when multicellularity arose, some cells made it, some didn’t, and those that didn’t probably suffered greatly. This time, we want everyone to get a ticket.
I appreciate and partly agree with this perspective, but I think you’re expressing it quite a bit too strongly.
As salient counterpoints, consider that nature repeatedly (in unrelated phyla) invented ganglia and brains, and that human-invented systems sometimes benefit from distribution and parallelism while other times benefiting from centralisation and vertical scaling.
So I certainly don’t think you can say that nature empirically strenuously asserts in either direction.
I do think you’re on the money saying
though of course a key question is whether it’s competitive (and under what conditions), and if not, what can be done about that (or what can be done to set conditions so that it is).
Another important question is whether (or under what conditions) it’s nice to be part of a greater whole. The kinds of primitive (?) greater wholes that humans currently get themselves into seem to vary in that regard, and one could imagine far greater variety there. Would it be nice to be a mitochondria-ish or gene-ish component of a larger being?
And one should note that, in the examples you’re giving, and perhaps in your described (if not conceived) vision, these are exclusive, hierarchical aggregations. (After all, my cells are part, exclusively, of me, and my mitochondria are part, exclusively, of a given cell, etc.)
In sharp contrast, contemporary human super-entities are very overlapping, as well as being porous, in ways which I think are very important and valuable (we often consider it a malaise if an aggregate is totalising and exclusionary, perhaps one of the bad markers of a cult—that fuzzy term).
I’d quibble, where this depends strongly on the interpretation of ‘suffered’, and ‘made it’!