How long until the earth gets eaten? 10th/50th/90th percentile: 3y, 12y, 37y.
Catastrophes induced by narrow capabilities (notably biotech) can push it further, so this might imply that they probably don’t occur.
No it doesn’t imply this, I set this disclaimer “Conditional on no strong governance success that effectively prevents basically all AI progress, and conditional on no huge global catastrophe happening in the meantime:”. Though yeah I don’t particularly expect those to occur.
The “AI might decide not to” point stands I think. This for me represents change of mind, I wouldn’t have previously endorsed this point, but since recently I think arbitrary superficial asks like this can become reflectively stable with nontrivial probability, resisting strong cost-benefit arguments even after intelligence explosion.
ok edited to sun. (i used earth first because i don’t know how long it will take to eat the sun, whereas earth seems likely to be feasible to eat quickly.)
(plausible to me that an aligned AI will still eat the earth but scan all the relevant information out of it and later maybe reconstruct it.)
No it doesn’t imply this, I set this disclaimer “Conditional on no strong governance success that effectively prevents basically all AI progress, and conditional on no huge global catastrophe happening in the meantime:”. Though yeah I don’t particularly expect those to occur.
The “AI might decide not to” point stands I think. This for me represents change of mind, I wouldn’t have previously endorsed this point, but since recently I think arbitrary superficial asks like this can become reflectively stable with nontrivial probability, resisting strong cost-benefit arguments even after intelligence explosion.
Right, I missed this.
ok edited to sun. (i used earth first because i don’t know how long it will take to eat the sun, whereas earth seems likely to be feasible to eat quickly.)
(plausible to me that an aligned AI will still eat the earth but scan all the relevant information out of it and later maybe reconstruct it.)