You’re adding constraints to the “humanity is a cancer” project which make it a lot harder. Why not settle for “wipe out humanity in a way that doesn’t cause much damage and let the planet heal itself”?
The idea of an anti-UFAI is intriguing. I’m not sure it’s much easier to design than an FAI.
I think the major barrier to the development of a “wipe out humans” UFAI is that the work would have to be done in secret.
It seems to me that an anti-UFAI that does not also prevent the creation of FAIs would, necessarily, be just as hard to make as an FAI. Identifying an FAI without having a sufficiently good model of what one is that you could make one seems implausible.
An anti-UFAI could have terms like ‘minimal collateral damage’ in it’s motivation that would cause it to prioritize stopping faster or more destructive AIs over slower or friendlier ones, voluntarily limit it’s own growth, accept ongoing human supervision, and cleanly self-destruct under appropriate circumstances.
An FAI is expected to make the world better, not just keep it from getting worse, and as such would need to be trusted with far more autonomy and long-term stability.
You’re adding constraints to the “humanity is a cancer” project which make it a lot harder. Why not settle for “wipe out humanity in a way that doesn’t cause much damage and let the planet heal itself”?
The idea of an anti-UFAI is intriguing. I’m not sure it’s much easier to design than an FAI.
I think the major barrier to the development of a “wipe out humans” UFAI is that the work would have to be done in secret.
It seems to me that an anti-UFAI that does not also prevent the creation of FAIs would, necessarily, be just as hard to make as an FAI. Identifying an FAI without having a sufficiently good model of what one is that you could make one seems implausible.
Am I wrong?
You’re at least plausible.
An anti-UFAI could have terms like ‘minimal collateral damage’ in it’s motivation that would cause it to prioritize stopping faster or more destructive AIs over slower or friendlier ones, voluntarily limit it’s own growth, accept ongoing human supervision, and cleanly self-destruct under appropriate circumstances.
An FAI is expected to make the world better, not just keep it from getting worse, and as such would need to be trusted with far more autonomy and long-term stability.