I would observe that any HEC computronium planet could be destroyed and replaced with a similar amount of computronium running more efficient non-HEC computations, supporting a much greater amount of flourishing and well-being. So the real question is, why suffer a huge utility hit to preserve a blackbox, which at its best is still much worse than your best, and at its worst is possibly truly astronomically dreadful?
There’s a game-theoretic component here as well: the choice to hide both encryption/decryption keys is not a neutral one. Any such civilization could choose to preserve at least limited access, and could also possibly provide verifiable proofs of what is going on inside (/gestures vaguely towards witness/functional encryption, PCP, and similar concepts). Since this is possible to some degree, choosing not to do so conveys information.
So, this suggests to me an unraveling argument: any such civilization which thinks its existence is ethically acceptable to all other civs will provide such proofs; any blackbox civ is then inferred to be one of the rest, with low average acceptability and so may be destroyed/replaced, so the civs which are ethically acceptable to almost all other civs will be better off providing the proof too; now the average blackbox civ is going to be even worse, so now the next most acceptable civ will want to be transparent and provide proof… And so on down to the point of civs so universally abhorrent that they are better off taking their chances as a blackbox rather than provide proof they should be burnt with fire. So you would then have good reason to expect any blackbox HEC civs you encounter to probably be one of the truly abominable ones.
why suffer a huge utility hit to preserve a blackbox, which at its best is still much worse than your best, and at its worst is possibly truly astronomically dreadful?
the reason i disagree with this is “killing people is bad” — i.e. i care more about satisfying the values of currently existing moral patients than satisfying the values of potential alternate moral patients; and those values can include “continuing to exist”. so if possible, even up to some reasonable compute waste factor, i’d want moral patients currently existing past event horizons to have their values satisfied.
as for the blackbox and universal abhorrence thing, i think that that smuggles in the assumptions “civilizations will tend to have roughly similar values” and “a civilization’s fate (such as being in an HEC without decryption keys) can be taken as representative of most of its inhabitants’ wills, let alone all”. that latter assumption especially, is evidenced against by the current expected fate of our own civilization (getting clipped).
I would observe that any HEC computronium planet could be destroyed and replaced with a similar amount of computronium running more efficient non-HEC computations, supporting a much greater amount of flourishing and well-being. So the real question is, why suffer a huge utility hit to preserve a blackbox, which at its best is still much worse than your best, and at its worst is possibly truly astronomically dreadful?
There’s a game-theoretic component here as well: the choice to hide both encryption/decryption keys is not a neutral one. Any such civilization could choose to preserve at least limited access, and could also possibly provide verifiable proofs of what is going on inside (/gestures vaguely towards witness/functional encryption, PCP, and similar concepts). Since this is possible to some degree, choosing not to do so conveys information.
So, this suggests to me an unraveling argument: any such civilization which thinks its existence is ethically acceptable to all other civs will provide such proofs; any blackbox civ is then inferred to be one of the rest, with low average acceptability and so may be destroyed/replaced, so the civs which are ethically acceptable to almost all other civs will be better off providing the proof too; now the average blackbox civ is going to be even worse, so now the next most acceptable civ will want to be transparent and provide proof… And so on down to the point of civs so universally abhorrent that they are better off taking their chances as a blackbox rather than provide proof they should be burnt with fire. So you would then have good reason to expect any blackbox HEC civs you encounter to probably be one of the truly abominable ones.
the reason i disagree with this is “killing people is bad” — i.e. i care more about satisfying the values of currently existing moral patients than satisfying the values of potential alternate moral patients; and those values can include “continuing to exist”. so if possible, even up to some reasonable compute waste factor, i’d want moral patients currently existing past event horizons to have their values satisfied.
as for the blackbox and universal abhorrence thing, i think that that smuggles in the assumptions “civilizations will tend to have roughly similar values” and “a civilization’s fate (such as being in an HEC without decryption keys) can be taken as representative of most of its inhabitants’ wills, let alone all”. that latter assumption especially, is evidenced against by the current expected fate of our own civilization (getting clipped).