I think the night-watchman concept is interesting, and probably is the ideal goal of alignment absent a good idea of what any other goal would ultimately lead to, but this post smuggles in concepts beyond the night watchman that would be very hard for anyone to swallow.
“Keeping the peace” internationally is pretty ambiguous, and I doubt that any major nation would be willing to give up the right of invasion as a last resort. Even if prevention of rogue super intelligence is seen as desirable, if preventing it also entails giving up a large amount of your current power, then I think world leaders will be pretty reluctant. The same can be said for “underhanded negotiation tactics”, which is both ambiguous, and not something most nations would want to give up. Most tactics in negotiation are underhanded in some sense, in that you’re using leverage you have over the other person to modify their actions.
The prevention of premature claims to space seems completely unnecessary as well. If the UK actually did something like sign over parts of Canada in return for a claim to the entire Milky Way, by now such a claim would be ignored completely (or maybe a small concession would be made for altering it) considering the UK has almost no space presence compared to the US, EU, China and Russia. The Treaty of Tordesillas was frequently renegotiated by Spain and Portugal, and almost completely ignored by the rest of the world.
Essentially I think this idea smuggles in a lot of other poison-pill, or unnecessary, ideas that would ultimately defeat the practically of implementing a night-watchman ASI at all. Either extinction is on the table, and we shouldn’t be giving ASI the power and drive to settle international conflicts or it isn’t, and we should be a lot more ambitious in the values and goals we assign.
I think the night-watchman concept is interesting, and probably is the ideal goal of alignment absent a good idea of what any other goal would ultimately lead to, but this post smuggles in concepts beyond the night watchman that would be very hard for anyone to swallow.
“Keeping the peace” internationally is pretty ambiguous, and I doubt that any major nation would be willing to give up the right of invasion as a last resort. Even if prevention of rogue super intelligence is seen as desirable, if preventing it also entails giving up a large amount of your current power, then I think world leaders will be pretty reluctant. The same can be said for “underhanded negotiation tactics”, which is both ambiguous, and not something most nations would want to give up. Most tactics in negotiation are underhanded in some sense, in that you’re using leverage you have over the other person to modify their actions.
The prevention of premature claims to space seems completely unnecessary as well. If the UK actually did something like sign over parts of Canada in return for a claim to the entire Milky Way, by now such a claim would be ignored completely (or maybe a small concession would be made for altering it) considering the UK has almost no space presence compared to the US, EU, China and Russia. The Treaty of Tordesillas was frequently renegotiated by Spain and Portugal, and almost completely ignored by the rest of the world.
Essentially I think this idea smuggles in a lot of other poison-pill, or unnecessary, ideas that would ultimately defeat the practically of implementing a night-watchman ASI at all. Either extinction is on the table, and we shouldn’t be giving ASI the power and drive to settle international conflicts or it isn’t, and we should be a lot more ambitious in the values and goals we assign.