We don’t want an ASI to be ”democratic”. We want it to be ”moral”. Many people in the West conflate the two words thinking that democratic and moral is the same thing but it is not. Democracy is a certain system of organizing a state. Morality is how people and (in the future) an ASI behave towards one another.
There are no obvious reasons why an authocratic state would care more or less about a future ASI being immoral, but an argument can be made that autocratic states will be more cautious and put more restrictions on the development of an ASI because autocrats usually fear any kind of opposition and an ASI could be a powerful adversary of itself or in the hands of powerful competitors.
I think “democratic” is often used to mean a system where everyone is given a meaningful (and roughly equal) weight into its decisions. People should probably use more precise language if that’s what they mean, but I do think it is often the implicit assumption.
And that quality is sort of prior to the meaning of “moral”, in that any weighted group of people (probably) defines a specific morality—according to their values, beliefs, and preferences. The morality of a small tribe may deem it as a matter of grave importance whether a certain rock has been touched by a woman, but barely anyone else truly cares (i.e. would still care if the tribe completely abandoned this position for endogenous reasons). A morality is more or less democratic to the extent that it weights everyone equally in this sense.
I’m not sure I buy that they will be more cautious in the context of an “arms race” with a foreign power. The Soviet Union took a lot of risks their bioweapons program during the cold war.
My impression is the CCP’s number one objective is preserving their own power over China. If they think creating ASI will help them with that, I fully expect them to pursue it (and in fact to make it their number one objective)
We don’t want an ASI to be ”democratic”. We want it to be ”moral”. Many people in the West conflate the two words thinking that democratic and moral is the same thing but it is not. Democracy is a certain system of organizing a state. Morality is how people and (in the future) an ASI behave towards one another.
There are no obvious reasons why an authocratic state would care more or less about a future ASI being immoral, but an argument can be made that autocratic states will be more cautious and put more restrictions on the development of an ASI because autocrats usually fear any kind of opposition and an ASI could be a powerful adversary of itself or in the hands of powerful competitors.
I think “democratic” is often used to mean a system where everyone is given a meaningful (and roughly equal) weight into its decisions. People should probably use more precise language if that’s what they mean, but I do think it is often the implicit assumption.
And that quality is sort of prior to the meaning of “moral”, in that any weighted group of people (probably) defines a specific morality—according to their values, beliefs, and preferences. The morality of a small tribe may deem it as a matter of grave importance whether a certain rock has been touched by a woman, but barely anyone else truly cares (i.e. would still care if the tribe completely abandoned this position for endogenous reasons). A morality is more or less democratic to the extent that it weights everyone equally in this sense.
I do want ASI to be “democratic” in this sense.
I’m not sure I buy that they will be more cautious in the context of an “arms race” with a foreign power. The Soviet Union took a lot of risks their bioweapons program during the cold war.
My impression is the CCP’s number one objective is preserving their own power over China. If they think creating ASI will help them with that, I fully expect them to pursue it (and in fact to make it their number one objective)