There are many versions of “night watchman” libertarianism, based on various ideas like “preventing the initiation of violence.” But when you try to make these ideas rigorous, you quickly run into thornt difficulties defining what it means to “initiate violence.” Is trespassing on private land violence? Is squatting violence? Your proposal operates at the level of nation-states, so equivalent questions might be, “How should the night watchman settle the issue of China and Taiwan? What happens if the Chinese government decides to send police officers to Taiwan to arrest people?” Or, “If a state becomes brutally oppressive to its citizens, who can intervene? What if a state starts committing a mass genocide against a regional ethnic group, and that group tries to secede?” Will the night watchman intervene? Will it allow other powers to intervene? It gets very hairy, very quickly.
You’re implicitly assuming that any kind of stable or reliable alignment is even possible. What if we live in a world where “aligning” an ASI is at least as difficult as (say) preventing teenagers from drinking, trying drugs, or getting each other pregnant? What if anything truly described as “intelligent” is intrinsically impossible to control in the long run?
Actually, I think both these issues might be variations of the same theme. The night watchman fails because you can’t truly describe international politics using simple and unambiguous rules with no loopholes. And I predict alignment will fail because you can’t describe what you want the AI to do using simple and unambiguous rules with no loopholes.
Or to look at it another way, every attempt to build an AI using clear cut rules and logic has failed completely. Every known example of intelligence is (approximately) billions of weights and non-linear thresholds, and the behavior of such a system only has a probability of doing what you want. Neither ethics nor intelligence can be reduced to syllogisms.
I see two major potential issues here:
There are many versions of “night watchman” libertarianism, based on various ideas like “preventing the initiation of violence.” But when you try to make these ideas rigorous, you quickly run into thornt difficulties defining what it means to “initiate violence.” Is trespassing on private land violence? Is squatting violence? Your proposal operates at the level of nation-states, so equivalent questions might be, “How should the night watchman settle the issue of China and Taiwan? What happens if the Chinese government decides to send police officers to Taiwan to arrest people?” Or, “If a state becomes brutally oppressive to its citizens, who can intervene? What if a state starts committing a mass genocide against a regional ethnic group, and that group tries to secede?” Will the night watchman intervene? Will it allow other powers to intervene? It gets very hairy, very quickly.
You’re implicitly assuming that any kind of stable or reliable alignment is even possible. What if we live in a world where “aligning” an ASI is at least as difficult as (say) preventing teenagers from drinking, trying drugs, or getting each other pregnant? What if anything truly described as “intelligent” is intrinsically impossible to control in the long run?
Actually, I think both these issues might be variations of the same theme. The night watchman fails because you can’t truly describe international politics using simple and unambiguous rules with no loopholes. And I predict alignment will fail because you can’t describe what you want the AI to do using simple and unambiguous rules with no loopholes.
Or to look at it another way, every attempt to build an AI using clear cut rules and logic has failed completely. Every known example of intelligence is (approximately) billions of weights and non-linear thresholds, and the behavior of such a system only has a probability of doing what you want. Neither ethics nor intelligence can be reduced to syllogisms.