Those who speak of FAI generally understand by it that we should be able to predict various things about what an AI will do: e.g., it will not bring about a future in which all that we hold dear is destroyed.
Clearly that’s difficult. It may be impossible. But your objection seems to apply equally to designing a chess-playing program with the intention that it will play much better chess than its creator, which is a thing that has been done successfully many times.
Those who speak of FAI generally understand by it that we should be able to predict various things about what an AI will do: e.g., it will not bring about a future in which all that we hold dear is destroyed.
Clearly that’s difficult. It may be impossible. But your objection seems to apply equally to designing a chess-playing program with the intention that it will play much better chess than its creator, which is a thing that has been done successfully many times.