Thanks for engaging. You point out one huge problem and some other lesser ones.
This isn’t a very motivating scenario. “We’re ruled by someone barely nice enough to not kill us” does not sound like victory.
One approach is to emphasize that we can put better people in charge and they can hand power to an aligned ASI as soon as it’s safe and we’re ready. Then frame the “barely good enough person” as a backup plan to emphasize that full success on societal alignment isn’t needed.
Yes, some decision-makers would die for their children. I don’t think it’s that many; that’s a pretty big ask. If they would, that’s great. Advocating for a positive vision of success at alignment isn’t in opposition to advocating for a pause. I think that we should stop building AGI right now, and that we may squeak by on alignment even if we don’t, if we get our shit together rapidly in some critical but not large ways.
It’s like saying “you should definitely not try that motorcycle jump but if you do you should really try to land it by doing these things, because it would be awesome if you did!” You can go ahead and say how awesome it would be if you didn’t try the jump until you’d practiced and planned enough, too.
Thanks for engaging. You point out one huge problem and some other lesser ones.
This isn’t a very motivating scenario. “We’re ruled by someone barely nice enough to not kill us” does not sound like victory.
One approach is to emphasize that we can put better people in charge and they can hand power to an aligned ASI as soon as it’s safe and we’re ready. Then frame the “barely good enough person” as a backup plan to emphasize that full success on societal alignment isn’t needed.
Yes, some decision-makers would die for their children. I don’t think it’s that many; that’s a pretty big ask. If they would, that’s great. Advocating for a positive vision of success at alignment isn’t in opposition to advocating for a pause. I think that we should stop building AGI right now, and that we may squeak by on alignment even if we don’t, if we get our shit together rapidly in some critical but not large ways.
It’s like saying “you should definitely not try that motorcycle jump but if you do you should really try to land it by doing these things, because it would be awesome if you did!” You can go ahead and say how awesome it would be if you didn’t try the jump until you’d practiced and planned enough, too.