How much of your plausibility-of-caring-slightly routes through this, vs the “actually just slightly intrinisically nice?” thing.
Maybe 60% trade, 40% intrinsic? Idk though.
I’m surprised at the “only 3 OOMs more expensive.” I haven’t done a calculation but that seems really implausible. Maybe I am not imagining how big an OOM is properly.
In particular, comparing “keep humans alive indefinitely, multigenerationally” vs “store them on ice without evening running them, turn them back on when/if you trade them.”
The dominant cost of “keeping humans alive physically” from a linear-returns+patient industrial expansion perspective is a small delay from rounding humans up into shelters (that you have to build and supply etc) or avoiding boiling the oceans (and other catastrophic environmental damage). The dominant cost of uploads is the small delay from rounding up and scanning people before they die. They both seem small, but the delays seem comparable.
Giving humans an entire star long term (e.g. after a few decades) is negligible in cost (like, <<10−20 of all galactic resources) relative to this delay (edit: for patient AIs which don’t prefer resources closer to earth), so I think keeping humans physically alive longer term is totally fine and just has higher upfront costs.
There’s also the cost of not killing humans as part of a takeover attempt and not using them for some other purpose (reasons (1) and (3)), but these are equal between uploads and physically keeping humans alive. I wasn’t intending to include this as I said “after AIs have fully solidified their power”, but if I did, then this makes the gap much smaller depending on how important reason (1) is.
It seems like a major early choice is “is the AI basically using the Earth to bootstrap the intergalactic probe process” or “is the AI only using the amount of Earth resources that leaves it mostly functional from human’s perspective, and then mostly using the rest of the planets.”
You note “seems like it’d only slow down the AI a bit, to first round up humans”. I haven’t done a calculation about it, but, seemed potentially like a very big deal to decide between “full steam ahead on beginning the dyson sphere using Earth Parts” vs “start the process from the moon and then mercury”, since there are harder-to-circumvent delays there.
You note “seems like it’d only slow down the AI a bit, to first round up humans”. I haven’t done a calculation about it, but, seemed potentially like a very big deal to decide between “full steam ahead on beginning the dyson sphere using Earth Parts” vs “start the process from the moon and then mercury”, since there are harder-to-circumvent delays there.
Can’t the AI proceed full steam ahead until it has enough industrial capacity to build shelters, then build shelters and put humans in the shelters (while pausing as needed at this point to avoid fatalities from environmental damage while humans are being rounded up), and then finish the industrial expansion (possibly upgrading shelters along the way as needed as the AI gets more resources)? Seems like naively this only delays you for as long as is needed to round up humans and up them in shelter which seems probably <1 year and probably <1 month. (At least if takeoff is pretty fast.)
Separately, not destroying the earth (and instead doing more of the growth in space) seems like it should cost <3 years of delay and probably <1 year of delay which is still pretty small as an absolute fraction of resources (for patient AIs). Like we’re talking 1 / billion or something.
I agree it’s small as a fraction of resources, but it still seems very expensive in terms of total resources since that’s a lotta galaxies falling outside the lightcone.
Maybe 60% trade, 40% intrinsic? Idk though.
The dominant cost of “keeping humans alive physically” from a linear-returns+patient industrial expansion perspective is a small delay from rounding humans up into shelters (that you have to build and supply etc) or avoiding boiling the oceans (and other catastrophic environmental damage). The dominant cost of uploads is the small delay from rounding up and scanning people before they die. They both seem small, but the delays seem comparable.
Giving humans an entire star long term (e.g. after a few decades) is negligible in cost (like, <<10−20 of all galactic resources) relative to this delay (edit: for patient AIs which don’t prefer resources closer to earth), so I think keeping humans physically alive longer term is totally fine and just has higher upfront costs.
There’s also the cost of not killing humans as part of a takeover attempt and not using them for some other purpose (reasons (1) and (3)), but these are equal between uploads and physically keeping humans alive. I wasn’t intending to include this as I said “after AIs have fully solidified their power”, but if I did, then this makes the gap much smaller depending on how important reason (1) is.
Mmm, nod. I maybe see it.
RE: “how long are you delaying?”
It seems like a major early choice is “is the AI basically using the Earth to bootstrap the intergalactic probe process” or “is the AI only using the amount of Earth resources that leaves it mostly functional from human’s perspective, and then mostly using the rest of the planets.”
You note “seems like it’d only slow down the AI a bit, to first round up humans”. I haven’t done a calculation about it, but, seemed potentially like a very big deal to decide between “full steam ahead on beginning the dyson sphere using Earth Parts” vs “start the process from the moon and then mercury”, since there are harder-to-circumvent delays there.
Can’t the AI proceed full steam ahead until it has enough industrial capacity to build shelters, then build shelters and put humans in the shelters (while pausing as needed at this point to avoid fatalities from environmental damage while humans are being rounded up), and then finish the industrial expansion (possibly upgrading shelters along the way as needed as the AI gets more resources)? Seems like naively this only delays you for as long as is needed to round up humans and up them in shelter which seems probably <1 year and probably <1 month. (At least if takeoff is pretty fast.)
Separately, not destroying the earth (and instead doing more of the growth in space) seems like it should cost <3 years of delay and probably <1 year of delay which is still pretty small as an absolute fraction of resources (for patient AIs). Like we’re talking 1 / billion or something.
I agree it’s small as a fraction of resources, but it still seems very expensive in terms of total resources since that’s a lotta galaxies falling outside the lightcone.