I really like where this post ended up. I skimmed the start and then went back to read it properly once I realised what the actual subject was.
Rather, their mistake is in believing that death absolves them from their duty to their children.
Personally I’m inclined to agree. Unfortunately, I think one of the things that the x-risk community doesn’t grapple with enough is that others can have very different moral premises. Some people are fine incurring a higher chance of risk for the whole future if it increases the chance that their ailing grandparents can get singularity-level medical care. Some people have irreconcilable disagreements about what values should govern the future. I don’t know what you’re meant to do with that.
Sincere advocacy for AI successionism makes AI safety research and policy all the more urgent
Again, personally agree as written, but I would guess that some successionists would reply with something like “your so-called safety research is mostly an attempt to impose your own narrow values; what I do may seem incomprehensible to you but that is because it is real safety research.”
You’re right, this is only a response to those who use successionism as an justification for blind accelerationism, not for all value disagreements. But it is a good argument against blind accelerationism: right now, we can’t be confident ASI will turn out to be aligned to any value system that isn’t defined tautologically.
I really like where this post ended up. I skimmed the start and then went back to read it properly once I realised what the actual subject was.
Personally I’m inclined to agree. Unfortunately, I think one of the things that the x-risk community doesn’t grapple with enough is that others can have very different moral premises. Some people are fine incurring a higher chance of risk for the whole future if it increases the chance that their ailing grandparents can get singularity-level medical care. Some people have irreconcilable disagreements about what values should govern the future. I don’t know what you’re meant to do with that.
Again, personally agree as written, but I would guess that some successionists would reply with something like “your so-called safety research is mostly an attempt to impose your own narrow values; what I do may seem incomprehensible to you but that is because it is real safety research.”
You’re right, this is only a response to those who use successionism as an justification for blind accelerationism, not for all value disagreements. But it is a good argument against blind accelerationism: right now, we can’t be confident ASI will turn out to be aligned to any value system that isn’t defined tautologically.