Climate change exists because doing something that’s bad for the world (carbon emission) is not priced. Climate change isn’t much worse than it is already because most people still can’t afford to live very climate unfriendly lives.
In this scenario, I’m mostly worried that without any constraints on what people can afford, not only might carbon emission go through the roof, but all other planetary boundaries that we know and don’t know yet might also be shattered. We could of course easily solve this problem by pricing externalities, which would not be very costly in an abundant world. Based on our track record, I just don’t think that we’ll do that.
Will we still have rainforest after the industrial explosion? Seems quite unlikely to me.
Will we still have rainforest after the industrial explosion? Seems quite unlikely to me.
This argument doesn’t stop at the biosphere or at the surface. By the same token, it shouldn’t be likely that we’ll have Earth or the Sun still remaining as celestial bodies, in an entirely literal sense. It might be possible in principle to decide not to disassemble them for fuel and raw materials, but also the direction pointed by the market argument is not entirely without merit.
Agree about the celestial bodies. Can you explain what you mean by “but also the direction pointed by the market argument is not entirely without merit”, and why the cited paper is relevant?
I would be reasonably optimistic if we had a democratic world government (or perhaps a UN-intent-aligned ASI blocking all other ASI) that we’d decide to leave at least some rainforest and the sun in one piece. I’m worried about international competition between states though where it becomes practically impossible due to such competition to not destroy earth for stuff. Maybe Russia will in the end win because it holds the greatest territory. Or more likely: the winning AI/industrial nation will conquer the rest of the world and will transform their earth to stuff as well.
Maybe we should have international treaties limiting the amount of nature a nation may convert to stuff?
There are whole (very distant) galaxies just at the boundary of practical reachability, that can be reached if colonization probes leave earlier, but never again if the probes get delayed. So there is a lot of value in getting this process underway a little earlier.
We lose around 1 / 10 billionth of the resources every year due to expansion. This is pretty negligible compared to potential differences in how well these resources end up being utilized (and other factors).
An additional hop to the nearby stars before starting the process would delay it by 10-50 years, which costs about 10 galaxies in expectation. This is somewhere between 1e8x and 1e14x more than the Solar System, depending on whether there is a way of using every part of the galaxy.
Mass is computation is people is value. Whether there is more than 1e8x-1e14x of diminishing returns in utility from additional galaxies after the first 4e9 galaxies is a question for aligned superphilosophers. I’m not making this call with any confidence, but I think it’s very plausible that marginal utility remains high.
I’m not seeing a tradeoff. If you speed things up by a few years, that’s also a few years earlier that local superintelligences get online at all of the stars in the reachable universe and start talking to each other at the speed of light, in particular propagating any globally applicable wisdom for the frontier of colonization, or observations made from star-sized telescopes and star-sized physics experiments, or conclusions reached by star-sized superintelligences, potentially making later hops of colonization more efficient.
So maybe launching drones to distant galaxies is not the appropriate first step in colonizing the universe, this doesn’t change the point that the Sun should still be eaten in order to take whatever step is actually more useful faster. Not eating the Sun at all doesn’t even result in producing Sun-sized value. It really does need to be quite valuable for its own sake, compared to the marginal galaxies, for leaving the Sun alone to be the better option.
It looks to me like this is a scenario where superhuman AI is intent-aligned. If that’s true, rainforests exist if humans prefer rainforests over mansions or superyachts or some other post-AGI luxury they could build from the same atoms. I’m afraid they won’t.
Climate change exists because doing something that’s bad for the world (carbon emission) is not priced. Climate change isn’t much worse than it is already because most people still can’t afford to live very climate unfriendly lives.
In this scenario, I’m mostly worried that without any constraints on what people can afford, not only might carbon emission go through the roof, but all other planetary boundaries that we know and don’t know yet might also be shattered. We could of course easily solve this problem by pricing externalities, which would not be very costly in an abundant world. Based on our track record, I just don’t think that we’ll do that.
Will we still have rainforest after the industrial explosion? Seems quite unlikely to me.
This argument doesn’t stop at the biosphere or at the surface. By the same token, it shouldn’t be likely that we’ll have Earth or the Sun still remaining as celestial bodies, in an entirely literal sense. It might be possible in principle to decide not to disassemble them for fuel and raw materials, but also the direction pointed by the market argument is not entirely without merit.
Agree about the celestial bodies. Can you explain what you mean by “but also the direction pointed by the market argument is not entirely without merit”, and why the cited paper is relevant?
I would be reasonably optimistic if we had a democratic world government (or perhaps a UN-intent-aligned ASI blocking all other ASI) that we’d decide to leave at least some rainforest and the sun in one piece. I’m worried about international competition between states though where it becomes practically impossible due to such competition to not destroy earth for stuff. Maybe Russia will in the end win because it holds the greatest territory. Or more likely: the winning AI/industrial nation will conquer the rest of the world and will transform their earth to stuff as well.
Maybe we should have international treaties limiting the amount of nature a nation may convert to stuff?
There are whole (very distant) galaxies just at the boundary of practical reachability, that can be reached if colonization probes leave earlier, but never again if the probes get delayed. So there is a lot of value in getting this process underway a little earlier.
We lose around 1 / 10 billionth of the resources every year due to expansion. This is pretty negligible compared to potential differences in how well these resources end up being utilized (and other factors).
An additional hop to the nearby stars before starting the process would delay it by 10-50 years, which costs about 10 galaxies in expectation. This is somewhere between 1e8x and 1e14x more than the Solar System, depending on whether there is a way of using every part of the galaxy.
Mass is computation is people is value. Whether there is more than 1e8x-1e14x of diminishing returns in utility from additional galaxies after the first 4e9 galaxies is a question for aligned superphilosophers. I’m not making this call with any confidence, but I think it’s very plausible that marginal utility remains high.
I’m not claiming that marginal utility is low, just that marginal utility is much higher for other things than speeding things up by a few years.
I’m not seeing a tradeoff. If you speed things up by a few years, that’s also a few years earlier that local superintelligences get online at all of the stars in the reachable universe and start talking to each other at the speed of light, in particular propagating any globally applicable wisdom for the frontier of colonization, or observations made from star-sized telescopes and star-sized physics experiments, or conclusions reached by star-sized superintelligences, potentially making later hops of colonization more efficient.
So maybe launching drones to distant galaxies is not the appropriate first step in colonizing the universe, this doesn’t change the point that the Sun should still be eaten in order to take whatever step is actually more useful faster. Not eating the Sun at all doesn’t even result in producing Sun-sized value. It really does need to be quite valuable for its own sake, compared to the marginal galaxies, for leaving the Sun alone to be the better option.
Computational ethics is false when applied to space colonization
This scenario is post superhuman AI. So rainforest exists iff the AI likes rainforest. Same goes for humans.
It looks to me like this is a scenario where superhuman AI is intent-aligned. If that’s true, rainforests exist if humans prefer rainforests over mansions or superyachts or some other post-AGI luxury they could build from the same atoms. I’m afraid they won’t.