I unfortunately think that if you are trying to prevent these two risks, then many interventions are close to diametrically opposed to what you would do if you tried to prevent loss of control risk. Indeed, my sense is these substantially mirror the motivations of the leading AI companies, which has resulted in an extremely intense race towards superintelligence in an attempt to permanently disempower “the bad guys” who would build the bioweapons or establish authoritarianism.[1]
It seems really important for people to know that loss of control doesn’t make the top of his list given that! (Though to be clear, there are lots of regulations one could pass that would help with these two and loss of control, but it indeed requires particular cool-headedness and integrity, which are not the dimensions on which I currently am most excited about Bores).
I think a slowdown on AI development is effective at reducing both AI-enabled authoritarianism and loss of control risks.
For bioweapons, it’s less clear, but probably still good to have some marginal slowdown. That said the optimal slodown timing might be pretty different from the other risks.
Another difference between preventing extreme loss of control / authoritarianism and reducing risk from bioweapons is that a pause on public deployment without pausing internal development/deployment is potentially good for bio stuff (though still dependeng on timing), while imo likely bad for the other risks.
Given that some form of slowdown/pause is perhaps the most commonly discussed and important intervention, saying that the required interventions are diametrically opposed seems wrong. Of course there are interventions besides slowdowns/pauses and some of these might have sharper disagreements between preventing loss of control and AI-enabled authoritarianism, though my guess is they are generally pretty correlated. I think diametrically opposed would still be wrong for AI-enabled bio risk in particular vs. loss of control, but I could better understand where youre coming from. For example, AI for epistemics/coordination interventions help with all 3 of these.
My guess is that you disagree with me about how similar the effects of these possible slowdowns/pauses are for loss of control vs. extreme AI-enabled authoritarianism. If so, would be curious as to why. It seems that allowing society more time to have more people more wake up and prevent extreme concentration of power is quite helpful.
I think a slowdown on AI development is effective at reducing both AI-enabled authoritarianism and loss of control risks.
[...]
My guess is that you disagree with me about how similar the effects of these possible slowdowns/pauses are for loss of control vs. extreme AI-enabled authoritarianism.
Maybe we are using words differently here. Almost always when people say “AI-enabled authoritarianism” excluding “loss of control” they mean “China might win the AI race and establish global authoritarianism”. I think a pause or any slowdown would currently marginally make China “more likely to win”.
I think there are many steps of confusion in the lines of argument people usually run here, but it appears to be a pretty central attractor. I agree with you that a sane attitude towards concentration of power would not think that the most likely way that happens is “because China wins the race”, for many different reasons, and that after you realize that, many of the good interventions become more correlated.
On bio: I think if you believe that we can build safe Superintelligence, at which point you almost certainly have bio-defense dominance, then you just want to get there as soon as possible, and any waiting or slowdown period makes someone using AI systems for bioterrorism, or some kind of national conflict escalating into bioweapons conflict more likely. More generally, I think the central driver of biorisk in a world where you aren’t worried about loss of control is foreign nations and terrorists getting access to the models, so marginal time in the period where bio is offense-dominant (as it is right now) is risky.
I am less confident on this bio picture. I do think most biorisk mitigations involve trying pretty hard to increase access to capabilities by “trusted” actors, and to be pretty hawkish about AI policy, coordination and cooperation. I think those attitudes are usually harmful to handling loss of control risks.
Maybe we are using words differently here. Almost always when people say “AI-enabled authoritarianism” excluding “loss of control” they mean “China might win the AI race and establish global authoritarianism”. I think a pause or any slowdown would currently marginally make China “more likely to win”.
Yup, we were imagining different definitions. I would bet that Bores’s concerns include extreme concentration of power in the US government / POTUS. @Eric Neyman feel free to clarify.
On bio: I think if you believe that we can build safe Superintelligence, at which point you almost certainly have bio-defense dominance, then you just want to get there as soon as possible, and any waiting or slowdown period makes someone using AI systems for bioterrorism, or some kind of national conflict escalating into bioweapons conflict more likely. More generally, I think the central driver of biorisk in a world where you aren’t worried about loss of control is foreign nations and terrorists getting access to the models, so marginal time in the period where bio is offense-dominant (as it is right now) is risky.
I think pausing now would be net good for reducing bio x-risk because it would allow more time to establish defenses before getting to more dangerous capabilities. I agree that once you hit a high enough capability level a pause starts being basd, nad this level probably happens significantly before loss of control PONR.
Appreciate the clarification! I am worried this is a temporary situation and would change if e.g. a democrat is elected next election. Separately, “include” is a relatively weak statement. Do you have any sense of how much of his concern about AI is driven by foreign governments developing powerful AI systems?
I don’t, sorry! Although, for what it’s worth, in the two conversations I’ve had with him about risks from AI, China didn’t come up.
Edited to add: also, for what it’s worth (and apologies for venturing into explicitly partisan politics), I am personally substantially more worried about AI-enabled authoritarianism under Trump than under most other possible presidents (though my worry definitely doesn’t go away entirely).
I think a slowdown on AI development is effective at reducing both AI-enabled authoritarianism and loss of control risks.
For bioweapons, it’s less clear, but probably still good to have some marginal slowdown. That said the optimal slodown timing might be pretty different from the other risks.
Another difference between preventing extreme loss of control / authoritarianism and reducing risk from bioweapons is that a pause on public deployment without pausing internal development/deployment is potentially good for bio stuff (though still dependeng on timing), while imo likely bad for the other risks.
Given that some form of slowdown/pause is perhaps the most commonly discussed and important intervention, saying that the required interventions are diametrically opposed seems wrong. Of course there are interventions besides slowdowns/pauses and some of these might have sharper disagreements between preventing loss of control and AI-enabled authoritarianism, though my guess is they are generally pretty correlated. I think diametrically opposed would still be wrong for AI-enabled bio risk in particular vs. loss of control, but I could better understand where youre coming from. For example, AI for epistemics/coordination interventions help with all 3 of these.
My guess is that you disagree with me about how similar the effects of these possible slowdowns/pauses are for loss of control vs. extreme AI-enabled authoritarianism. If so, would be curious as to why. It seems that allowing society more time to have more people more wake up and prevent extreme concentration of power is quite helpful.
Maybe we are using words differently here. Almost always when people say “AI-enabled authoritarianism” excluding “loss of control” they mean “China might win the AI race and establish global authoritarianism”. I think a pause or any slowdown would currently marginally make China “more likely to win”.
I think there are many steps of confusion in the lines of argument people usually run here, but it appears to be a pretty central attractor. I agree with you that a sane attitude towards concentration of power would not think that the most likely way that happens is “because China wins the race”, for many different reasons, and that after you realize that, many of the good interventions become more correlated.
On bio: I think if you believe that we can build safe Superintelligence, at which point you almost certainly have bio-defense dominance, then you just want to get there as soon as possible, and any waiting or slowdown period makes someone using AI systems for bioterrorism, or some kind of national conflict escalating into bioweapons conflict more likely. More generally, I think the central driver of biorisk in a world where you aren’t worried about loss of control is foreign nations and terrorists getting access to the models, so marginal time in the period where bio is offense-dominant (as it is right now) is risky.
I am less confident on this bio picture. I do think most biorisk mitigations involve trying pretty hard to increase access to capabilities by “trusted” actors, and to be pretty hawkish about AI policy, coordination and cooperation. I think those attitudes are usually harmful to handling loss of control risks.
Yup, we were imagining different definitions. I would bet that Bores’s concerns include extreme concentration of power in the US government / POTUS. @Eric Neyman feel free to clarify.
I think pausing now would be net good for reducing bio x-risk because it would allow more time to establish defenses before getting to more dangerous capabilities. I agree that once you hit a high enough capability level a pause starts being basd, nad this level probably happens significantly before loss of control PONR.
I don’t speak for Bores, but I think Eli is correct.
Appreciate the clarification! I am worried this is a temporary situation and would change if e.g. a democrat is elected next election. Separately, “include” is a relatively weak statement. Do you have any sense of how much of his concern about AI is driven by foreign governments developing powerful AI systems?
I don’t, sorry! Although, for what it’s worth, in the two conversations I’ve had with him about risks from AI, China didn’t come up.
Edited to add: also, for what it’s worth (and apologies for venturing into explicitly partisan politics), I am personally substantially more worried about AI-enabled authoritarianism under Trump than under most other possible presidents (though my worry definitely doesn’t go away entirely).