It’s not obvious to me that it takes 80+ years to get double-digit alignment success probabilities, from where we are. Waiting a few decades strikes me as obviously smart from a selfish perspective; e.g., AGI in 2052 is a lot selfishly better than AGI in 2032, if you’re under age 50 today.
But also, I think the current state of humanity’s alignment knowledge is very bad. I think your odds of surviving into the far future are a lot higher if you die in a few decades and get cryopreserved and then need to hope AGI works out in 80+ years, than if you survive to see AGI in the next 20 years.
True, you can make use of the Gompertz curve to get marginal benefit from waiting a bit while you still have a low marginal probability of non-AGI death.
So we only need to worry about researchers who have lower estimates of unaligned AGI causing their death, or who think that AGI is a long way out and want to hurry it up now.
Unfortunately, cryopreservation isn’t nearly as reliable as needed in order to assume immortality is achieved. While we’ve gotten better at it, it still relies on toxic chemicals in order to vitrify the brain.
It’s not obvious to me that it takes 80+ years to get double-digit alignment success probabilities, from where we are. Waiting a few decades strikes me as obviously smart from a selfish perspective; e.g., AGI in 2052 is a lot selfishly better than AGI in 2032, if you’re under age 50 today.
But also, I think the current state of humanity’s alignment knowledge is very bad. I think your odds of surviving into the far future are a lot higher if you die in a few decades and get cryopreserved and then need to hope AGI works out in 80+ years, than if you survive to see AGI in the next 20 years.
True, you can make use of the Gompertz curve to get marginal benefit from waiting a bit while you still have a low marginal probability of non-AGI death.
So we only need to worry about researchers who have lower estimates of unaligned AGI causing their death, or who think that AGI is a long way out and want to hurry it up now.
Unfortunately, cryopreservation isn’t nearly as reliable as needed in order to assume immortality is achieved. While we’ve gotten better at it, it still relies on toxic chemicals in order to vitrify the brain.
I’m not saying it’s reliable!! I’m saying the odds of alignment success in the next 20 years currently looks even worse.