If there was a button that would kill me with a 60% probability and transport me into a utopia for billions of years with a 15% probability, I would feel very scared to press that button
This is because the correct answer is option three: try to modify the button to lower the 60 and raise the 15, until such time as a 1-in-5 chance of survival is a net improvement relative to your default situation. I’d be much more likely to press that button if I’d just jumped out of an airplane without a parachute. Or if there was a hundred mile wide asteroid near-guaranteed to hit Earth next Tuesday.
Also, this is the first year where the people close to me are cognizant enough of AI that I can talk to them about life plan derailment expectations and not be dismissed as crazy. I can tell my parents to try to really attend to their health more than they have in the past, and why. I can explain to my wife that hey, we should both expect start surfing a wave of frequent job changes until the concept of a job stops making sense. It’s been honestly very freeing to be able to discuss these things somewhere other than this community. I’m still a little hesitant to openly talk to my sisters about what their children’s futures might look like, but even that is starting to change.
This is because the correct answer is option three: try to modify the button to lower the 60 and raise the 15, until such time as a 1-in-5 chance of survival is a net improvement relative to your default situation.
Yes, the counterfactual I was imagining in this button world was just living a normal life and dying at the end. If indeed there’s a way to shift around the probabilities I’d devote my life to it. Which is what we’re doing!
It’s been honestly very freeing to be able to discuss these things somewhere other than this community.
I agree. This year I’ve had the policy of being very direct about what I think about crazy AI futures even with people outside of the AI safety community. I held a powerpoint presentation to my close family members talking about AGI and AI safety and how the world is going to be crazy in the coming decades. When my relatives ask me about having kids, I say “By the time I’d have had kids, if humanity is even around, who knows what the concept of kids will look like. Maybe we’ll be growing them in vats. Maybe we’ll all be uploaded.”
Of course, I don’t say all of that every time. Most of the time people aren’t in the mood for those sorts of discussions. But people have started taking these arguments more seriously as AI has had more and more of an effect and appeared more and more in the news.
No, (at least for men) it takes much longer than 10 months to make a kid in a way that’s worth doing. You have to find a partner willing to do it with you, which takes an unpredictable amount of time. (I guess if you’re rich you can hire an egg donor and a surrogate.)
This is because the correct answer is option three: try to modify the button to lower the 60 and raise the 15, until such time as a 1-in-5 chance of survival is a net improvement relative to your default situation. I’d be much more likely to press that button if I’d just jumped out of an airplane without a parachute. Or if there was a hundred mile wide asteroid near-guaranteed to hit Earth next Tuesday.
Also, this is the first year where the people close to me are cognizant enough of AI that I can talk to them about life plan derailment expectations and not be dismissed as crazy. I can tell my parents to try to really attend to their health more than they have in the past, and why. I can explain to my wife that hey, we should both expect start surfing a wave of frequent job changes until the concept of a job stops making sense. It’s been honestly very freeing to be able to discuss these things somewhere other than this community. I’m still a little hesitant to openly talk to my sisters about what their children’s futures might look like, but even that is starting to change.
Yes, the counterfactual I was imagining in this button world was just living a normal life and dying at the end. If indeed there’s a way to shift around the probabilities I’d devote my life to it. Which is what we’re doing!
I agree. This year I’ve had the policy of being very direct about what I think about crazy AI futures even with people outside of the AI safety community. I held a powerpoint presentation to my close family members talking about AGI and AI safety and how the world is going to be crazy in the coming decades. When my relatives ask me about having kids, I say “By the time I’d have had kids, if humanity is even around, who knows what the concept of kids will look like. Maybe we’ll be growing them in vats. Maybe we’ll all be uploaded.”
Of course, I don’t say all of that every time. Most of the time people aren’t in the mood for those sorts of discussions. But people have started taking these arguments more seriously as AI has had more and more of an effect and appeared more and more in the news.
It only takes 10 months to make one…
Would have != could have.
No, (at least for men) it takes much longer than 10 months to make a kid in a way that’s worth doing. You have to find a partner willing to do it with you, which takes an unpredictable amount of time. (I guess if you’re rich you can hire an egg donor and a surrogate.)