I think part of the reason why the US government is acting so early is to reduce the risk of a coup led by an AI company and its management. There is a subcategory of scenarios where AI is nominally “aligned” to the AI company’s CEO and will follow the orders of that specific individual. Once AI grows sufficiently powerful and advanced, the CEO uses the AI to seize power and subject the fate of humanity to their whims. This is unlikely to end well.
By assuming direct control over AI development, the US government will be able to reduce the chance of this bad outcome happening. Other bad outcomes, such as human extinction or permanent enslavement by ASI, become significantly more likely as the government will select for agents that are more willing to commit violence and select for (feigned) obedience. Hence, I think the net effect of early government intervention similar to what is happening right now is approximately net-neutral.
The US government acting so early increase the risk of a coup led by high-ranking politicians and military officials. There is a subcategory of scenarios where AI is nominally “aligned” to e.g. the President or Secretary of Defense and will follow the orders of that specific individual. Once AI grows sufficiently powerful and advanced, they will use the AI to seize power and subject the fate of humanity to their whims. This is unlikely to end well.
By assuming direct control over AI development, the US government will be able to increase the chance of this bad outcome happening. Other bad outcomes, such as human extinction or permanent enslavement by ASI, also become significantly more likely as the government will select for agents that are more willing to commit violence and select for (feigned) obedience. Hence, I think the net effect of early government intervention similar to what is happening right now is net-negative on several counts.
I have two questions and long winded explanations associated with them.
First, why do you think the US government will be better at creating “alignment” relative to privately controlled corporations?
My ASI timelines are rather long, with my median date for ASI arrival lying in the late 2030s. I do not foresee an end to America’s political instability and believe that America’s government will cycle between Democrat and Republican control on 4 year intervals. Since the red tribe and the blue tribe are locked in a cold civil war against each other, efforts started by a Republican administration will be ruthlessly sabotaged by a Democrat administration and vice versa. This self sabotaging behavior will render the already difficult task of “alignment” (remember, for this to occur true alignment must be solved) much harder as much of the progress made by one administration will be thrown out by the next.
Meanwhile, the four major AI companies in the United States, Open AI, Anthropic, Google and XAI have relatively consistent leadership involving an entrenched CEO who will likely remain in control up until a rouge superintelligence emerges. Compared to the top brass of an “alignment” effort run by the United States government, these CEOs have a much better understanding of their models and can enact plans over significantly longer timescales without fear of replacement. Hence, a CEO is more likely to devise and successfully enact a plan that successfully “aligns” the models their company creates.
Second, why do you think the danger posed by successful “alignment” of an AI to a faction of the US government is greater than the danger imposed by “alignment” to a CEO?
I believe that the actions of the US government are constrained by prestige. Both major factions in the US government are slightly incentivized to share the wealth an “aligned” superintelligence creates because they derive a small amount of legitimacy from a wealthy and happy population. Since the cost of harvesting Universal Basic Income from the efforts of an “aligned” superintelligence is rather low, the government would be willing to spend pittances to make itself look better. Meanwhile, CEOs are often maximally ruthless, willing to increase their personal wealth and power through their company without a care for the consequences. This makes genocidal catastrophes, where an “aligned” superintelligence is used to kill most people and enslave the rest so one individual can claim all of Earth’s resources, more likely. Essentially, I believe that futures where a superintelligence is “aligned” to a CEO are generally worse than futures where a superintelligence is “aligned” to the US government.
Even if AI capabilities stalled, I would still be at the very least uncertain about whether they will still be free and fair elections in 2028.
In any case I expect substantial organizational continuity to persist in the military-industrial complex in particular, comparable to that in major AI companies.
I expect AI CEOs to be somewhat less likely to be malevolent, and much less likely to be ideological fanatics, than politicans and military officials.
Uncensored models being available to a self-selected elite, and the rest of us getting whatever those elites decide to give us after censorship is more dangerous than giving uncensored models to everyone. AI gatekeeping in the guise of “safety” is going to lead to tangible immediate harms.
I think part of the reason why the US government is acting so early is to reduce the risk of a coup led by an AI company and its management. There is a subcategory of scenarios where AI is nominally “aligned” to the AI company’s CEO and will follow the orders of that specific individual. Once AI grows sufficiently powerful and advanced, the CEO uses the AI to seize power and subject the fate of humanity to their whims. This is unlikely to end well.
By assuming direct control over AI development, the US government will be able to reduce the chance of this bad outcome happening. Other bad outcomes, such as human extinction or permanent enslavement by ASI, become significantly more likely as the government will select for agents that are more willing to commit violence and select for (feigned) obedience. Hence, I think the net effect of early government intervention similar to what is happening right now is approximately net-neutral.
The US government acting so early increase the risk of a coup led by high-ranking politicians and military officials. There is a subcategory of scenarios where AI is nominally “aligned” to e.g. the President or Secretary of Defense and will follow the orders of that specific individual. Once AI grows sufficiently powerful and advanced, they will use the AI to seize power and subject the fate of humanity to their whims. This is unlikely to end well.
By assuming direct control over AI development, the US government will be able to increase the chance of this bad outcome happening. Other bad outcomes, such as human extinction or permanent enslavement by ASI, also become significantly more likely as the government will select for agents that are more willing to commit violence and select for (feigned) obedience. Hence, I think the net effect of early government intervention similar to what is happening right now is net-negative on several counts.
I have two questions and long winded explanations associated with them.
First, why do you think the US government will be better at creating “alignment” relative to privately controlled corporations?
My ASI timelines are rather long, with my median date for ASI arrival lying in the late 2030s. I do not foresee an end to America’s political instability and believe that America’s government will cycle between Democrat and Republican control on 4 year intervals. Since the red tribe and the blue tribe are locked in a cold civil war against each other, efforts started by a Republican administration will be ruthlessly sabotaged by a Democrat administration and vice versa. This self sabotaging behavior will render the already difficult task of “alignment” (remember, for this to occur true alignment must be solved) much harder as much of the progress made by one administration will be thrown out by the next.
Meanwhile, the four major AI companies in the United States, Open AI, Anthropic, Google and XAI have relatively consistent leadership involving an entrenched CEO who will likely remain in control up until a rouge superintelligence emerges. Compared to the top brass of an “alignment” effort run by the United States government, these CEOs have a much better understanding of their models and can enact plans over significantly longer timescales without fear of replacement. Hence, a CEO is more likely to devise and successfully enact a plan that successfully “aligns” the models their company creates.
Second, why do you think the danger posed by successful “alignment” of an AI to a faction of the US government is greater than the danger imposed by “alignment” to a CEO?
I believe that the actions of the US government are constrained by prestige. Both major factions in the US government are slightly incentivized to share the wealth an “aligned” superintelligence creates because they derive a small amount of legitimacy from a wealthy and happy population. Since the cost of harvesting Universal Basic Income from the efforts of an “aligned” superintelligence is rather low, the government would be willing to spend pittances to make itself look better. Meanwhile, CEOs are often maximally ruthless, willing to increase their personal wealth and power through their company without a care for the consequences. This makes genocidal catastrophes, where an “aligned” superintelligence is used to kill most people and enslave the rest so one individual can claim all of Earth’s resources, more likely. Essentially, I believe that futures where a superintelligence is “aligned” to a CEO are generally worse than futures where a superintelligence is “aligned” to the US government.
Not only I have shorter ASI timelines, I think the AI capabilities required for authoritarianism to be lower than ASI, are already there to some extent, and will already be far more advanced by 2028.
Even if AI capabilities stalled, I would still be at the very least uncertain about whether they will still be free and fair elections in 2028.
In any case I expect substantial organizational continuity to persist in the military-industrial complex in particular, comparable to that in major AI companies.
I expect AI CEOs to be somewhat less likely to be malevolent, and much less likely to be ideological fanatics, than politicans and military officials.
Uncensored models being available to a self-selected elite, and the rest of us getting whatever those elites decide to give us after censorship is more dangerous than giving uncensored models to everyone. AI gatekeeping in the guise of “safety” is going to lead to tangible immediate harms.