I don’t know whether what I am saying concurs with the ‘usual LW approach,’ but I would very quickly move past the option of killing most people.
If we currently present ourselves with two options (letting dangerous UFAI projects progress and killing lots of people), then we should not grimace and take whichever choice we deem slightly more palatable—we should instead seek a third alternative.
In my eyes, this is what I have done—shutting down AGI projects would not necessitate the killing of large numbers of people, and perhaps a third alternative could be found to killing even one. To maintain that the premise “rapidly self-improving UFAI will almost certainly kill us all, if created” leads to killing most people, you must explain why, indeed, killing most people would reduce the existential risk presented by UFAI significantly more than would completely shutting down UFAI projects.
I don’t know whether what I am saying concurs with the ‘usual LW approach,’ but I would very quickly move past the option of killing most people.
If we currently present ourselves with two options (letting dangerous UFAI projects progress and killing lots of people), then we should not grimace and take whichever choice we deem slightly more palatable—we should instead seek a third alternative.
In my eyes, this is what I have done—shutting down AGI projects would not necessitate the killing of large numbers of people, and perhaps a third alternative could be found to killing even one. To maintain that the premise “rapidly self-improving UFAI will almost certainly kill us all, if created” leads to killing most people, you must explain why, indeed, killing most people would reduce the existential risk presented by UFAI significantly more than would completely shutting down UFAI projects.
Edit: For clarification purposes, I do not believe that shutting down UFAI projects is the best use of my time. The above discussion refers to a situation in which people are much closer to creating UFAI than FAI and will continue to be given expected rate of progress.