I agree with this, but at that point philosophical dilemmas don’t matter for alignment purposes, because from an alignment perspective you only need to (ideally) have agreement between the AI and human on the answers, and the answers to philosophical dilemmas don’t actually matter (at least for value-laden philosophical dilemmas).
I agree with Eliezer on the hard part of the alignment problem:
So philosophy being hard is not a valid argument for AI alignment being hard.
That said, I do think philosophical dilemmas like this (as well as trying to steer us towards moral-trade futures) is a big reason I’ve come to think of space governance as important, neglected and (for now) much more tractable than basically everything else with the same level of importance or higher.
More generally, one very important priority is to move ourselves away from a regime where whoever makes a unilateral claim and seizes land in space owns it, and we will need to prohibit interstellar settlement for a time (because otherwise due to light-speed limitations, it becomes effectively impossible for the central government to catch up with them).
Thankfully, this is far easier than ending the AI race for now.
I agree with this, but at that point philosophical dilemmas don’t matter for alignment purposes, because from an alignment perspective you only need to (ideally) have agreement between the AI and human on the answers, and the answers to philosophical dilemmas don’t actually matter (at least for value-laden philosophical dilemmas).
I agree with Eliezer on the hard part of the alignment problem:
Ebenzer Dukais’s comment
So philosophy being hard is not a valid argument for AI alignment being hard.
That said, I do think philosophical dilemmas like this (as well as trying to steer us towards moral-trade futures) is a big reason I’ve come to think of space governance as important, neglected and (for now) much more tractable than basically everything else with the same level of importance or higher.
More generally, one very important priority is to move ourselves away from a regime where whoever makes a unilateral claim and seizes land in space owns it, and we will need to prohibit interstellar settlement for a time (because otherwise due to light-speed limitations, it becomes effectively impossible for the central government to catch up with them).
Thankfully, this is far easier than ending the AI race for now.