Maybe the result of one person’s clones forming a very capable Em Collective would still be suboptimal and undemocratic from the perspective of the rest of humanity, but it wouldn’t kill everyone, and I think wouldn’t lead to especially bad outcomes if you start from the right person.
I think the risk of a homogeneous collective of many instances of a single person’s consciousness is more serious than “suboptimal and undemocratic” suggests. Even assuming you could find a perfectly well-intentioned person to clone, identical minds share the same blindspots and biases. Without diversity of perspective, even earnestly benevolent ideas could—and I imagine would—lead to unintented catastrophe.
I also wonder how you would identify the right person, as I can’t think of anyone I would trust with that degree of power.
Would someone who legitimately, deeply believes lack of diversity of perspective would be catastrophic, and who values avoiding that catastrophe and thus will in fact take rapid, highest-priority action to get as close as possible to democratically constructed values and collectively rational insight, be able to avoid this problem?
I also wonder how you would identify the right person, as I can’t think of anyone I would trust with that degree of power.
I agree, and I think it’s worse than OP believes in a way similar to how you do: my impression is that one of the mechanisms by which power corrupts is that even someone well intentioned typically has difficulty thinking clearly about tradeoffs when those tradeoffs are measured in terms of “lives you, personally, made the sole decision to intentionally end in favor of other lives”, and non-conflict scenarios like health or resource construction produce those sorts of binds. More briefly: it’s psychologically difficult to both care and have power.
Also, many people are already corrupt and seek power by dishonestly appearing to be safe with power, which seems like a more common reason for power to appear to corrupt: they were simply already corrupt.
Would someone who legitimately, deeply believes lack of diversity of perspective would be catastrophic, and who values avoiding that catastrophe and thus will in fact take rapid, highest-priority action to get as close as possible to democratically constructed values and collectively rational insight, be able to avoid this problem?
No, I don’t think anyone could, barring the highly unlikely case of a superintelligence perfectly aligned with human values (and even still, maybe not—human values are inconsistent and contradictory). Also, I think a system of democratically constructed values would probably be at odds with rational insight, unfortunately.
Regarding the rest, agreed. Heading into verbotten-ish political territory here, but see also Jenny Holzer and Chomsky on this.
I think the risk of a homogeneous collective of many instances of a single person’s consciousness is more serious than “suboptimal and undemocratic” suggests. Even assuming you could find a perfectly well-intentioned person to clone, identical minds share the same blindspots and biases. Without diversity of perspective, even earnestly benevolent ideas could—and I imagine would—lead to unintented catastrophe.
I also wonder how you would identify the right person, as I can’t think of anyone I would trust with that degree of power.
Would someone who legitimately, deeply believes lack of diversity of perspective would be catastrophic, and who values avoiding that catastrophe and thus will in fact take rapid, highest-priority action to get as close as possible to democratically constructed values and collectively rational insight, be able to avoid this problem?
I agree, and I think it’s worse than OP believes in a way similar to how you do: my impression is that one of the mechanisms by which power corrupts is that even someone well intentioned typically has difficulty thinking clearly about tradeoffs when those tradeoffs are measured in terms of “lives you, personally, made the sole decision to intentionally end in favor of other lives”, and non-conflict scenarios like health or resource construction produce those sorts of binds. More briefly: it’s psychologically difficult to both care and have power.
Also, many people are already corrupt and seek power by dishonestly appearing to be safe with power, which seems like a more common reason for power to appear to corrupt: they were simply already corrupt.
No, I don’t think anyone could, barring the highly unlikely case of a superintelligence perfectly aligned with human values (and even still, maybe not—human values are inconsistent and contradictory). Also, I think a system of democratically constructed values would probably be at odds with rational insight, unfortunately.
Regarding the rest, agreed. Heading into verbotten-ish political territory here, but see also Jenny Holzer and Chomsky on this.