No one cares because...there are other systems who are not operating on a complete set of human values (including many small, relatively dumb AI systems) that are steering the world instead?
No one cares for the same reason no one (in the present) cares that AI can now pass the Turing Test. By the time we get there, we are grappling with different questions.
“Can you define an AI model that preserves a particular definition of human values under iterative self-improvement” simply won’t be seen as the question of the day, because by the time we can do it, it will feel “obvious” or “unimpressive”.
No one cares because...there are other systems who are not operating on a complete set of human values (including many small, relatively dumb AI systems) that are steering the world instead?
No one cares for the same reason no one (in the present) cares that AI can now pass the Turing Test. By the time we get there, we are grappling with different questions.
“Can you define an AI model that preserves a particular definition of human values under iterative self-improvement” simply won’t be seen as the question of the day, because by the time we can do it, it will feel “obvious” or “unimpressive”.