Indeed, moral philosophy is the main bottleneck for AI progress. Two thoughts:
I’ve come to the conclusion that there can be no “complete” system of morality – only a method akin to the scientific method. Just as claims about nature can be disproven or reinterpreted, moral claims must always remain open to revision in light of changing circumstances. I’m currently working on such a method.
Even a perfectly rational moral system would fail to align AI with people fully, because humans can be deeply irrational. Therefore, no form of moral perfection can be relied upon in principle.
Indeed, moral philosophy is the main bottleneck for AI progress.
Two thoughts:
I’ve come to the conclusion that there can be no “complete” system of morality – only a method akin to the scientific method. Just as claims about nature can be disproven or reinterpreted, moral claims must always remain open to revision in light of changing circumstances. I’m currently working on such a method.
Even a perfectly rational moral system would fail to align AI with people fully, because humans can be deeply irrational. Therefore, no form of moral perfection can be relied upon in principle.