Freedom is not necessarily freedom to get eaten by a recursively self-improving AI you built that then proceeds to eat everyone else. But freedom under negotiated norms that are mostly about placing boundaries on concerns that each party is primarily in control of might look like cosmopolitanism, not very sensitive to details of individual terminal values, with no need for merging of values in private spaces.
sounds maybe kinda like a utopia design i’ve previously come up with, where you get your private computational garden and all interactions are voluntary.
that said some values need to come interfere into people’s gardens: you can’t create arbitrarily suffering moral patients, you might have to in some way be stopped from partaking in some molochianisms, etc.
Hence the misaligned ASI example, the private spaces shouldn’t be allowed arbitrary computation, they still need to respect some norms. Individual control is only retained over what the norms leave out, which might allow almost all details of individual terminal values to be expressed, as long as they don’t step on what the norms proscribe. The boundaries set by norms/commitments are about possible computations/behaviors, not particularly about physical location.
Freedom is not necessarily freedom to get eaten by a recursively self-improving AI you built that then proceeds to eat everyone else. But freedom under negotiated norms that are mostly about placing boundaries on concerns that each party is primarily in control of might look like cosmopolitanism, not very sensitive to details of individual terminal values, with no need for merging of values in private spaces.
sounds maybe kinda like a utopia design i’ve previously come up with, where you get your private computational garden and all interactions are voluntary.
that said some values need to come interfere into people’s gardens: you can’t create arbitrarily suffering moral patients, you might have to in some way be stopped from partaking in some molochianisms, etc.
Hence the misaligned ASI example, the private spaces shouldn’t be allowed arbitrary computation, they still need to respect some norms. Individual control is only retained over what the norms leave out, which might allow almost all details of individual terminal values to be expressed, as long as they don’t step on what the norms proscribe. The boundaries set by norms/commitments are about possible computations/behaviors, not particularly about physical location.