most people today do not appreciate that future computational-life forms will be just as meaningful as the meat-based ones today, and should not sacrifice orders of magnitudes more life-years than will be lived on Earth for the difference between a big ball of plasma and something else that recreates the same quality of light.
huh i agree with this, but i’ve been imagining that earth and many galaxies will be in the domain of preferences which want to “live on actual planets” because of this chain-of-logic:
the lightcone is very much larger than just earth. (wikipedia says “[there are] an estimated 100 billion [galaxies] in all of the observable universe”). we’d want to give up earth (and some surrounding many galaxies) in return for more good possible futures, because of the good which can be derived from the non-earth parts of those ones.
some beings care disproportionately about what happens to earth and its sun. some of them are alignment researchers (e.g. some comments in this thread), or otherwise influencing the trajectory
it’s better for collaboration if those preferences determine the fate of earth and its sun. i.e. this prevents some values from being incentivized to compete for alignment to them in particular (to the detriment of general success rates).
i can write more about this part if wanted. notably (unless i’ve made a mistake) it doesn’t rely on trust, but that’s probably not very clear with just this.
actually, this case is less interesting than the general case i had in mind. because in this case one side has non-linearity in value of this sun versus others, both sides want to make this trade even without a greater chance of both dying if they don’t.
huh i agree with this, but i’ve been imagining that earth and many galaxies will be in the domain of preferences which want to “live on actual planets” because of this chain-of-logic:
the lightcone is very much larger than just earth. (wikipedia says “[there are] an estimated 100 billion [galaxies] in all of the observable universe”). we’d want to give up earth (and some surrounding many galaxies) in return for more good possible futures, because of the good which can be derived from the non-earth parts of those ones.
some beings care disproportionately about what happens to earth and its sun. some of them are alignment researchers (e.g. some comments in this thread), or otherwise influencing the trajectory
it’s better for collaboration if those preferences determine the fate of earth and its sun. i.e. this prevents some values from being incentivized to compete for alignment to them in particular (to the detriment of general success rates).
i can write more about this part if wanted. notably (unless i’ve made a mistake) it doesn’t rely on trust, but that’s probably not very clear with just this.
actually, this case is less interesting than the general case i had in mind. because in this case one side has non-linearity in value of this sun versus others, both sides want to make this trade even without a greater chance of both dying if they don’t.