I’ve thought a bit about ideas like this, and talked to much smarter people than myself about such ideas—and they usually dismiss them, which I take as a strong signal this may be a misguided idea.
I think the Machiavellian model of politics is largely correct—and it just is the case that if you look closely at any great change in policy you see, beneath the idealized narrative, a small coterie of very smart ideologues engaging in Machiavellian politics.
To the extent overt political power is necessary for EA causes to succeed, Machiavellian politics will be necessary and good. However, this sort of duplicitous regulatory judo you advocate strikes me as possibly backfiring—by politicizing AI in this way those who are working on the actually important AI safety research become very tempting targets to the mechanisms you hope to summon. We see hints of this already.
To the extent it is possible to get people with correct understanding of the actually important problem in positions of bureaucratic and moral authority, this seems really, really good. Machiavellian politics will be required to do this. Such people may indeed need to lie about their motivations. And perhaps they may find it necessary to manipulate the population in the way you describe.
However, if you don’t have such people actually in charge and use judo mind tricks to manipulate existing authorities to bring AI further into the culture war, you are summoning a beast you, by definition, lack the power to tame.
I suspect it would backfire horribly, incentivize safety washing of various kinds in existing organizations who are best positioned to shape regulation, making new alignment orgs like Conjecture and Redwood very difficult to start, and worst of all making overtly caring about the actual problem very politically difficult
> I’ve thought a bit about ideas like this, and talked to much smarter people than myself about such ideas—and they usually dismiss them, which I take as a strong signal this may be a misguided idea.
I honestly don’t know whether slowing down AI progress in these ways is/isn’t a good idea. It seems plausibly good to me. I do think I disagree about whether the “much smarter people”s dismissal of these ideas is a strong signal.
Why I disagree about the strong signal thing:
I had to push through some fear as I wrote the sentence about it seeming plausibly good to me, because as I wrote it I imagined a bunch of potential conflict between e.g. AI safety folks, and AI folks. For example, a few months ago I asked a safety researcher within a major AI lab what they thought of e.g. people saying they thought it was bad to do/speed AI research. They gave me an expression I interpreted as fear, swallowed, and said something like: gosh, it was hard to think about because it might lead to conflict between them and their colleagues at the AI lab.
At least one person well-versed in AI safety, who I personally think non-stupid about policy also, has told me privately that they think it’s probably helpful if people-at-large decide to try to slow AI or to talk about AI being scary, but that it seems disadvantageous (for a number of good, altruistic reasons) for them personally to do it.
Basically, it seems plausible to me that:
1. There’s a “silence of elites” on the topic of “maybe we should try by legal and non-violent means to slow AI progress, e.g. by noticing aloud that maybe it is anti-social for people to be hurtling toward the ability to kill everyone,”
2. Lots of people (such as Trevor1 in the parent comment) interpret this silence as evidence that the strategy
3. But it is actually mostly evidence that many elites are in local contexts where their personal ability to do the specific work they are trying to do would be harmed by them saying such things aloud, plus social contagion/mimicry of such views.
I am *not* sure the above 1-3 is the case. I have also talked with folks who’ve thought a lot about safety and who honestly think that existential risk is lower if we have AI soon (before humanity can harm itself in other ways), for example. But I think the above is plausible enough that I’d recommend being pretty careful not to interpret elite silence as necessarily meaning there’s a solid case against “slow down AI,” and pushing instead for inside-view arguments that actually make sense to you, or to others who you think are good thinkers and who are not politically entangled with AI or AI safety.
When I try it myself on an inside view, I see things pointing in multiple directions. Would love to see LW try to hash it out.
All of this makes sense, and I do agree that it’s worth consideration (I quadruple upvoted the check mark on your comment). Mainly in-person conversations, since the absolute worst case scenario with in-person conversations is that new people learn a ton of really good information about the nitty-gritty problems with mass public outreach; such as international affairs. I don’t know if there’s a knowable upper bound on how wayward/compromised/radicalized this discussion could get if such discussion takes place predominantly on the internet.
I’d also like to clarify that I’m not “interpreting this silence as evidence”, I’ve talked to AI policy people, and I also am one, and I understand the details of why we reflexively shoot down the idea of mass public outreach. It all boils down to ludicrously powerful, territorial, invisible people with vested interests in AI, and zero awareness of what AGI is or why it might be important (for the time being).
I have also talked with folks who’ve thought a lot about safety and who honestly think that existential risk is lower if we have AI soon (before humanity can harm itself in other ways), for example.
It seems hard to make the numbers come out that way. E.g. suppose human-level AGI in 2030 would cause a 60% chance of existential disaster and a 40% chance of existential disaster becoming impossible, and human-level AGI in 2050 would cause a 50% chance of existential disaster and a 50% chance of existential disaster becoming impossible. Then to be indifferent about AI timelines, conditional on human-level AGI in 2050, you’d have to expect a 1⁄5 probability of existential disaster from other causes in the 2030-2050 period. (That way, with human-level AGI in 2050, you’d have a 1⁄2 * 4⁄5 = 40% chance of surviving, just like with human-level AGI in 2030.) I don’t really know of non-AI risks in the ballpark of 10% per decade.
(My guess at MIRI people’s model is more like 99% chance of existential disaster from human-level AGI in 2030 and 90% in 2050, in which case indifference would require a 90% chance of some other existential disaster in 2030-2050, to cut 10% chance of survival down to 1%.)
Well written post that will hopefully stir up some good discussion :)
My impression is that LW/EA people prefer to avoid conflict, and when conflict is necessary don’t want to use misleading arguments/tactics (with BS regulations seen as such).
To be frank, I think the main reason this approach hasn’t been strongly tried is because most AI alignment enthusiasts are asocial nerds, who don’t know how to / are uncomfortable with trying to emotionally manipulate people in the ways needed for successful anti-AI propaganda. We need a ruthless PR team as effective as a presidential campaign staff, badly.
I’d also like to point out that most of the major players here are big tech companies which a large portion of the population already distrusts, and conservatives in particular could easily be swayed against them by leveraging their anti-elitist and anti-academic sentiments. But, again, most AI alignment enthusiasts are liberals and do not necessarily understand or respect the intelligence of conservatives, who are the ones most likely to actually recognize the danger and care, because conserving the status quo is literally what they do. Suppose we got Trump supporters on our side. This thought probably makes you itch; but it would pack a punch.
The main problem there, of course, is that the moment conservatives become anti something, liberals react by becoming even more vehemently pro it, and vice versa. I’m not sure how to navigate that, but we can’t simply ignore half the world and their potential support.
Prototype conservative rant:
We all know that Silicon Valley elites and their Russian and Chinese comrades use social media platforms to callously manipulate our children, turn honest, hardworking Americans against each other, and destroy the foundations of our democracy, all to pad their pockets.
But did you know they’re also working to build artificial general intelligences—unfeeling mockeries of humanity, with no goal except to maximize profit, regardless of what innocent people they have to step on? If these criminal scum are allowed to continue playing God and successfully create some kind of sentient computer life, who knows what kinds of demonic forces could be unleashed upon our world?
It’s our responsibility as patriots to oppose these companies every way we can, and use the full force of our democracy—and the market, voting with our dollars—to remind them to think twice before treading on the American people.
The current situation is almost exactly analogous to the creation of the atomic bomb during World War 2.
It seems that the correct behavior in that case was not to worry at all, since the doomsday predictions never came to fruition, and now the bomb has faded out of public consciousness.
Overall, I think slowing research for any reason is misguided, especially in a field as important as AI. If you did what you’re saying in this post, you would also delay progress on many extremely positive developments like
Drug discovery
Automation of unpleasant jobs
Human intelligence augmentation
Automated theorem proving
Self-driving cars
Etc, etc
And those things are more clearly inevitable and very likely coming sooner than a godlike, malicious AGI.
Think about everything we would have missed out on if you had put this plan into action a few decades ago. There would be no computer vision, no DALLE-2, no GPT-3. You would have given up so much, and you would not have prevented anything bad from happening.
Thanks for the post—I think there are some ways heavy regulation of AI could be very counterproductive or ineffective for safety:
If AI progress slows down enough in countries were safety-concerned people are especially influential, then these countries (and their companies) will fall behind in international AI development. This would eliminate much/most of safety-concerned people’s opportunities for impacting how AI goes.
If China “catches up” to the US in AI (due to US over-regulation) when AI is looking increasingly economically and militarily important, that could easily motivate US lawmakers to hit the gas on AI (which would at least undo some of the earlier slowing down of AI, and would at worst spark an international race to the bottom on AI).
Also, you mention,
The community strategy (insofar as there even is one) is to bet everything on getting a couple of technical alignment folks onto the team at top research labs in the hopes that they will miraculously solve alignment before the mad scientists in the office next door turn on the doomsday machine.
From conversation, my understanding is some governance/policy folks fortunately have (somewhat) more promising ideas than that. (This doesn’t show up much on this site, partly because: people on here tend to not be as interested in governance, these professionals tend to be busy, the ideas are fairly rough, and getting the optics right can be more important for governance ideas.) I hear there’s some work aimed at posting about some of these ideas—until then, chatting with people (e.g., by reaching out to people at conferences) might be the best way to learn about these ideas.
Hire think tanks to write white papers about the harm caused by poorly designed, quickly deployed AI systems.
Take advantage of the inevitable fuck-ups and disasters caused by narrow AI to press leaders for tighter regulations and more bureaucracy
These two ideas seem the closest to being reasonable strategies. I’d like to see think tanks prepare now for what to recommend if something triggers public demand for doing something about AI.
Strongly tying AI safety to wokeness in the mind of the public seems like a high-risk strategy, especially with the possibility of a backlash against wokism in the US—the baby might be thrown out with the bathwater.
I share your concerns, as do some people I know. I also feel like thoughts along these lines have become more common in the community over the past few months, but nobody I was aware of had yet tried to take steps to do something about it and take the first steps to try and galvanise a new effort on AGI risk outreach.
We’re not sure yet what we’re aiming for exactly, but we all think that current efforts at communicating AGI risk to the wider world are lacking, and we need AGI danger to be widely understood and politically planned for, like climate change is.
Personally, I’d like for our efforts to end up creating something like a new umbrella EA org specialised in wide outreach on AGI risk, from grand strategy conceptualising to planning and carrying out concrete campaigns to networking existing outreach efforts.
This isn’t 2012. AGI isn’t as far mode as it used to be. You can show people systems like PaLM, and how good their language understanding and reasoning capabilities are getting. I think that while it may be very hard, normal people could indeed be made to see that this is going somewhere dangerous, so long as their jobs do not directly depend on believing otherwise.
{{Blockquote | text=Recruit more EAs from China to join this project there (particularly those with high-level connections in the CCP)}}
I straight up don’t believe this is going to work. Trying to influence the Chinese government into restricting their own access to technology that is of the utmost geopolitical importance seems about as hard as AI alignment to me. I don’t want to say it’s impossible, but it’s close. Without absolute international cooperation on the issue we’re just shifting the problem into countries and research communities that are intransparent to almost all the leading AI alignment researchers right now.
Screaming loudly: “Hey, people in the government trying to mitigate this problem, could you please put in place ‘stupid regulations’ to slow down AI?”
Yes, thank you for loudly shouting to the world that AI safety regulations are “stupid” and are just to slow down AI progress, exactly the kind of messaging we need.
Humans still don’t seem to care much about the minimization of harm among all their foolish goals.
Why not crush such selfish animals?
Conscious beings that fail to criticize their own evolved “alignment” aren’t worth preserving, extinction would be a mercy.
I’ve thought a bit about ideas like this, and talked to much smarter people than myself about such ideas—and they usually dismiss them, which I take as a strong signal this may be a misguided idea.
I think the Machiavellian model of politics is largely correct—and it just is the case that if you look closely at any great change in policy you see, beneath the idealized narrative, a small coterie of very smart ideologues engaging in Machiavellian politics.
To the extent overt political power is necessary for EA causes to succeed, Machiavellian politics will be necessary and good. However, this sort of duplicitous regulatory judo you advocate strikes me as possibly backfiring—by politicizing AI in this way those who are working on the actually important AI safety research become very tempting targets to the mechanisms you hope to summon. We see hints of this already.
To the extent it is possible to get people with correct understanding of the actually important problem in positions of bureaucratic and moral authority, this seems really, really good. Machiavellian politics will be required to do this. Such people may indeed need to lie about their motivations. And perhaps they may find it necessary to manipulate the population in the way you describe.
However, if you don’t have such people actually in charge and use judo mind tricks to manipulate existing authorities to bring AI further into the culture war, you are summoning a beast you, by definition, lack the power to tame.
I suspect it would backfire horribly, incentivize safety washing of various kinds in existing organizations who are best positioned to shape regulation, making new alignment orgs like Conjecture and Redwood very difficult to start, and worst of all making overtly caring about the actual problem very politically difficult
> I’ve thought a bit about ideas like this, and talked to much smarter people than myself about such ideas—and they usually dismiss them, which I take as a strong signal this may be a misguided idea.
I honestly don’t know whether slowing down AI progress in these ways is/isn’t a good idea. It seems plausibly good to me. I do think I disagree about whether the “much smarter people”s dismissal of these ideas is a strong signal.
Why I disagree about the strong signal thing:
I had to push through some fear as I wrote the sentence about it seeming plausibly good to me, because as I wrote it I imagined a bunch of potential conflict between e.g. AI safety folks, and AI folks. For example, a few months ago I asked a safety researcher within a major AI lab what they thought of e.g. people saying they thought it was bad to do/speed AI research. They gave me an expression I interpreted as fear, swallowed, and said something like: gosh, it was hard to think about because it might lead to conflict between them and their colleagues at the AI lab.
At least one person well-versed in AI safety, who I personally think non-stupid about policy also, has told me privately that they think it’s probably helpful if people-at-large decide to try to slow AI or to talk about AI being scary, but that it seems disadvantageous (for a number of good, altruistic reasons) for them personally to do it.
Basically, it seems plausible to me that:
1. There’s a “silence of elites” on the topic of “maybe we should try by legal and non-violent means to slow AI progress, e.g. by noticing aloud that maybe it is anti-social for people to be hurtling toward the ability to kill everyone,”
2. Lots of people (such as Trevor1 in the parent comment) interpret this silence as evidence that the strategy
3. But it is actually mostly evidence that many elites are in local contexts where their personal ability to do the specific work they are trying to do would be harmed by them saying such things aloud, plus social contagion/mimicry of such views.
I am *not* sure the above 1-3 is the case. I have also talked with folks who’ve thought a lot about safety and who honestly think that existential risk is lower if we have AI soon (before humanity can harm itself in other ways), for example. But I think the above is plausible enough that I’d recommend being pretty careful not to interpret elite silence as necessarily meaning there’s a solid case against “slow down AI,” and pushing instead for inside-view arguments that actually make sense to you, or to others who you think are good thinkers and who are not politically entangled with AI or AI safety.
When I try it myself on an inside view, I see things pointing in multiple directions. Would love to see LW try to hash it out.
All of this makes sense, and I do agree that it’s worth consideration (I quadruple upvoted the check mark on your comment). Mainly in-person conversations, since the absolute worst case scenario with in-person conversations is that new people learn a ton of really good information about the nitty-gritty problems with mass public outreach; such as international affairs. I don’t know if there’s a knowable upper bound on how wayward/compromised/radicalized this discussion could get if such discussion takes place predominantly on the internet.
I’d also like to clarify that I’m not “interpreting this silence as evidence”, I’ve talked to AI policy people, and I also am one, and I understand the details of why we reflexively shoot down the idea of mass public outreach. It all boils down to ludicrously powerful, territorial, invisible people with vested interests in AI, and zero awareness of what AGI is or why it might be important (for the time being).
It seems hard to make the numbers come out that way. E.g. suppose human-level AGI in 2030 would cause a 60% chance of existential disaster and a 40% chance of existential disaster becoming impossible, and human-level AGI in 2050 would cause a 50% chance of existential disaster and a 50% chance of existential disaster becoming impossible. Then to be indifferent about AI timelines, conditional on human-level AGI in 2050, you’d have to expect a 1⁄5 probability of existential disaster from other causes in the 2030-2050 period. (That way, with human-level AGI in 2050, you’d have a 1⁄2 * 4⁄5 = 40% chance of surviving, just like with human-level AGI in 2030.) I don’t really know of non-AI risks in the ballpark of 10% per decade.
(My guess at MIRI people’s model is more like 99% chance of existential disaster from human-level AGI in 2030 and 90% in 2050, in which case indifference would require a 90% chance of some other existential disaster in 2030-2050, to cut 10% chance of survival down to 1%.)
Agreed. Maybe the blocker here is that LW/EA people don’t have many contacts in public policy, and are much more familiar with tech.
Well written post that will hopefully stir up some good discussion :)
My impression is that LW/EA people prefer to avoid conflict, and when conflict is necessary don’t want to use misleading arguments/tactics (with BS regulations seen as such).
To be frank, I think the main reason this approach hasn’t been strongly tried is because most AI alignment enthusiasts are asocial nerds, who don’t know how to / are uncomfortable with trying to emotionally manipulate people in the ways needed for successful anti-AI propaganda. We need a ruthless PR team as effective as a presidential campaign staff, badly.
I’d also like to point out that most of the major players here are big tech companies which a large portion of the population already distrusts, and conservatives in particular could easily be swayed against them by leveraging their anti-elitist and anti-academic sentiments. But, again, most AI alignment enthusiasts are liberals and do not necessarily understand or respect the intelligence of conservatives, who are the ones most likely to actually recognize the danger and care, because conserving the status quo is literally what they do. Suppose we got Trump supporters on our side. This thought probably makes you itch; but it would pack a punch.
The main problem there, of course, is that the moment conservatives become anti something, liberals react by becoming even more vehemently pro it, and vice versa. I’m not sure how to navigate that, but we can’t simply ignore half the world and their potential support.
Prototype conservative rant:
It seems that the correct behavior in that case was not to worry at all, since the doomsday predictions never came to fruition, and now the bomb has faded out of public consciousness.
Overall, I think slowing research for any reason is misguided, especially in a field as important as AI. If you did what you’re saying in this post, you would also delay progress on many extremely positive developments like
Drug discovery
Automation of unpleasant jobs
Human intelligence augmentation
Automated theorem proving
Self-driving cars
Etc, etc
And those things are more clearly inevitable and very likely coming sooner than a godlike, malicious AGI.
Think about everything we would have missed out on if you had put this plan into action a few decades ago. There would be no computer vision, no DALLE-2, no GPT-3. You would have given up so much, and you would not have prevented anything bad from happening.
Thanks for the post—I think there are some ways heavy regulation of AI could be very counterproductive or ineffective for safety:
If AI progress slows down enough in countries were safety-concerned people are especially influential, then these countries (and their companies) will fall behind in international AI development. This would eliminate much/most of safety-concerned people’s opportunities for impacting how AI goes.
If China “catches up” to the US in AI (due to US over-regulation) when AI is looking increasingly economically and militarily important, that could easily motivate US lawmakers to hit the gas on AI (which would at least undo some of the earlier slowing down of AI, and would at worst spark an international race to the bottom on AI).
Also, you mention,
From conversation, my understanding is some governance/policy folks fortunately have (somewhat) more promising ideas than that. (This doesn’t show up much on this site, partly because: people on here tend to not be as interested in governance, these professionals tend to be busy, the ideas are fairly rough, and getting the optics right can be more important for governance ideas.) I hear there’s some work aimed at posting about some of these ideas—until then, chatting with people (e.g., by reaching out to people at conferences) might be the best way to learn about these ideas.
These two ideas seem the closest to being reasonable strategies. I’d like to see think tanks prepare now for what to recommend if something triggers public demand for doing something about AI.
Strongly tying AI safety to wokeness in the mind of the public seems like a high-risk strategy, especially with the possibility of a backlash against wokism in the US—the baby might be thrown out with the bathwater.
I share your concerns, as do some people I know. I also feel like thoughts along these lines have become more common in the community over the past few months, but nobody I was aware of had yet tried to take steps to do something about it and take the first steps to try and galvanise a new effort on AGI risk outreach.
So we did: https://forum.effectivealtruism.org/posts/DS3frSuoNynzvjet4/agi-safety-communications-initiative
We’re not sure yet what we’re aiming for exactly, but we all think that current efforts at communicating AGI risk to the wider world are lacking, and we need AGI danger to be widely understood and politically planned for, like climate change is.
Personally, I’d like for our efforts to end up creating something like a new umbrella EA org specialised in wide outreach on AGI risk, from grand strategy conceptualising to planning and carrying out concrete campaigns to networking existing outreach efforts.
This isn’t 2012. AGI isn’t as far mode as it used to be. You can show people systems like PaLM, and how good their language understanding and reasoning capabilities are getting. I think that while it may be very hard, normal people could indeed be made to see that this is going somewhere dangerous, so long as their jobs do not directly depend on believing otherwise.
If you’d like to join up, do follow the link.
I have been saying similar things and really wish we had gotten started sooner. It’s a very promising approach.
{{Blockquote | text=Recruit more EAs from China to join this project there (particularly those with high-level connections in the CCP)}}
I straight up don’t believe this is going to work. Trying to influence the Chinese government into restricting their own access to technology that is of the utmost geopolitical importance seems about as hard as AI alignment to me. I don’t want to say it’s impossible, but it’s close. Without absolute international cooperation on the issue we’re just shifting the problem into countries and research communities that are intransparent to almost all the leading AI alignment researchers right now.
Screaming loudly: “Hey, people in the government trying to mitigate this problem, could you please put in place ‘stupid regulations’ to slow down AI?”
Yes, thank you for loudly shouting to the world that AI safety regulations are “stupid” and are just to slow down AI progress, exactly the kind of messaging we need.
Humans still don’t seem to care much about the minimization of harm among all their foolish goals. Why not crush such selfish animals? Conscious beings that fail to criticize their own evolved “alignment” aren’t worth preserving, extinction would be a mercy.