I am sorry for your loss. Death is natural but it is so, so bad.
I’m assuming you’re posting this here in part to foster a discussion of this tension between preventing deaths in the short term and taking more risks on killing everyone by getting alignment wrong if we rush toward AGI.
This tension is probably going to become more widespread as the concept of AGI becomes more prominent. Many people will want faster progress, in hopes of saving themselves or their loved ones. Longtermism is pretty dominant here on LW, but it is very much a minority view in society at large. Thus, this urge to rush will have to be countered by spreading an awareness of how rushing toward AGI is increasing the odds for older people, while risking the lives of their children and grandchildren. And all of the glorious generations to follow—while most people aren’t longtermist, the idea of unimaginable flourishing does hold some weight in their minds, and it isn’t that ahrd to imagine in broad form.
I face this dilemma myself. At 50 and in imperfect health, my likely end falls somewhere in the middle of my predicted range of hitting longevity takeoff. Any small speedup or slowdown might shift my odds substantially. I don’t know what I’d do if I had real power over our rate of progress, but I don’t. So I’ll continue advocating that we slow down as much as we can, while also working as fast as we can to align our first AGI/ASIs. That speed will improve our odds of collective survival in the likely case that we can’t slow down substantially. And it might even save a few more of the precious unique minds now alive.
I am sorry for your loss. Death is natural but it is so, so bad.
I’m assuming you’re posting this here in part to foster a discussion of this tension between preventing deaths in the short term and taking more risks on killing everyone by getting alignment wrong if we rush toward AGI.
This tension is probably going to become more widespread as the concept of AGI becomes more prominent. Many people will want faster progress, in hopes of saving themselves or their loved ones. Longtermism is pretty dominant here on LW, but it is very much a minority view in society at large. Thus, this urge to rush will have to be countered by spreading an awareness of how rushing toward AGI is increasing the odds for older people, while risking the lives of their children and grandchildren. And all of the glorious generations to follow—while most people aren’t longtermist, the idea of unimaginable flourishing does hold some weight in their minds, and it isn’t that ahrd to imagine in broad form.
I face this dilemma myself. At 50 and in imperfect health, my likely end falls somewhere in the middle of my predicted range of hitting longevity takeoff. Any small speedup or slowdown might shift my odds substantially. I don’t know what I’d do if I had real power over our rate of progress, but I don’t. So I’ll continue advocating that we slow down as much as we can, while also working as fast as we can to align our first AGI/ASIs. That speed will improve our odds of collective survival in the likely case that we can’t slow down substantially. And it might even save a few more of the precious unique minds now alive.