That’s interesting—basically here we’re trying to educate an AI into human values, but human values are going to swiftly be changed to something different (and bad from our perspective).
I think there’s no magical solution—either we build a FAI properly (which is very very hard), and it would stop the third party, or we have an AI that we value load and try and prevent our values from changing while it’s happening.
The omission/commission thing applies to value loading AIs, not to traditional FAI. But I admit it’s not the best analogy.
I’m not sure commission/omission distinction is really the key here. This becomes clearer by inverting the situation a bit:
Some third party is about to forcibly wirehead all of humanity. How should your moral agent reason about whether to intervene and prevent this?
That’s interesting—basically here we’re trying to educate an AI into human values, but human values are going to swiftly be changed to something different (and bad from our perspective).
I think there’s no magical solution—either we build a FAI properly (which is very very hard), and it would stop the third party, or we have an AI that we value load and try and prevent our values from changing while it’s happening.
The omission/commission thing applies to value loading AIs, not to traditional FAI. But I admit it’s not the best analogy.