While i agree with this post almost entirely, i can’t pretend that i still feel like we have no choice. Trying to stop the development of AI today, is kind of like trying to have stopped the manhattan project. Even worse in my opinion if i think about it: the profit incentive is there, the military incentive is there. And from history we’ve seen that public disapproval doesn’t necessarily stop governments or corporations from anything, if the incentive is large enough. The whole world is already essentially operating as a large machine with only one goal: maximize profits. And it does so no matter the cost, which is why we have more slaves today than ever before in history. All that being said, it can’t hurt to try our best. But i also cannot pretend the feeling of hopelessness is not there, and that it’s not (at least to my current understanding) well justified. Whenever we think of cases where something changed because of public disapproval, it was likely because the consequences of that disapproval outweighed the benefits of continuing. But with AI, that’s simply not the case.
The mechanism isn’t [public disapproval] → [AI moratorium]. That sounds hopeless to me, too. What I’m actually relatively optimistic about is [public awareness] → [educating policymakers] + [signal to policymakers that they are allowed to do something about it] → [common knowledge of the desire to act] → [AI moratorium]. This feels very doable to me after getting back from speaking with my congressional offices in DC.
The key here is that most politicians don’t want to take these risks either. Really very few people are on board with the whole “create superintelligence we don’t know how to control or make care about us” thing. If enough politicians and policymakers learn that the “create superintelligence” thing necessarily brings a risk of human extinction, one would hope that they would act out of their own fear and desire for self-preservation. But because of how politics works (or doesn’t), especially in the US, it takes a lot to overcome the strong social fear of speaking out. They need to know that their constituents have their backs, as opposed to either supporting the status quo or having very different priorities.
So public disapproval isn’t a lever with which to move the whole world. It’s just a firm tap to send the balance in the direction of action. The whole mechanism is being built simultaneously in parallel, and the education and advocacy work gets easier the more the public is engaged with the issue. The bigger the tap, the smaller the gap.
Trying to stop the development of AI today, is kind of like trying to have stopped the manhattan project.
You are correct that stopping the Manhatten project would have been much easier.
It was a secret project, and the incentives for most of the decision-makers involved were around producing “better-but-accountable” outcomes. There were multiple individuals to whom if you presented a sufficiently plausible argument, might have been able to stop the Manhattan project, up until the actual attack on Hiroshima. Fermi et al. had a pretty good chance of preventing the bomb from being used, and had other technical experts agreed that the risks were too high, I think the Manhattan project could have been stopped.
The current situation is much worse, with far more people having strong incentives to deceive others, themselves, and knowingly take risks.
All of that said, I still don’t think this means we have no choice. There are multiple ways of coordinating to solve a bad equilibrium, and I don’t think the incentives for pursuing AI are nearly so universal.
Most of the public recognizes that they have a lot of incentive to stop the development of ASI, even if right now it is more focused on job losses, they also recognize that the gains of AI are unlikely to be distributed evenly, and most Americans in particular care more about their chance of gaining in social status, than almost any other terminal outcome of policy (I can provide some citations if needed.)
More importantly, that public disapproval likely has a stable equilibrium in that they probably should be worried about the existential risk, whatever percentage it really is.
All that needs to follow is that the public disapproval needs to be strong enough and translate into a strong enough signal that policy makers are duly incentivize to take the concerns seriously, and policy make.
That isn’t super easy, but it isn’t pure fantasy at all. Everyone can see how the incentives for AI firms favor total race dynamics, but for most of humanity and even most Americans incentives are more than strong enough to prefer caution.
While i agree with this post almost entirely, i can’t pretend that i still feel like we have no choice. Trying to stop the development of AI today, is kind of like trying to have stopped the manhattan project. Even worse in my opinion if i think about it: the profit incentive is there, the military incentive is there. And from history we’ve seen that public disapproval doesn’t necessarily stop governments or corporations from anything, if the incentive is large enough. The whole world is already essentially operating as a large machine with only one goal: maximize profits. And it does so no matter the cost, which is why we have more slaves today than ever before in history. All that being said, it can’t hurt to try our best. But i also cannot pretend the feeling of hopelessness is not there, and that it’s not (at least to my current understanding) well justified. Whenever we think of cases where something changed because of public disapproval, it was likely because the consequences of that disapproval outweighed the benefits of continuing. But with AI, that’s simply not the case.
The mechanism isn’t [public disapproval] → [AI moratorium]. That sounds hopeless to me, too. What I’m actually relatively optimistic about is [public awareness] → [educating policymakers] + [signal to policymakers that they are allowed to do something about it] → [common knowledge of the desire to act] → [AI moratorium]. This feels very doable to me after getting back from speaking with my congressional offices in DC.
The key here is that most politicians don’t want to take these risks either. Really very few people are on board with the whole “create superintelligence we don’t know how to control or make care about us” thing. If enough politicians and policymakers learn that the “create superintelligence” thing necessarily brings a risk of human extinction, one would hope that they would act out of their own fear and desire for self-preservation. But because of how politics works (or doesn’t), especially in the US, it takes a lot to overcome the strong social fear of speaking out. They need to know that their constituents have their backs, as opposed to either supporting the status quo or having very different priorities.
So public disapproval isn’t a lever with which to move the whole world. It’s just a firm tap to send the balance in the direction of action. The whole mechanism is being built simultaneously in parallel, and the education and advocacy work gets easier the more the public is engaged with the issue. The bigger the tap, the smaller the gap.
You are correct that stopping the Manhatten project would have been much easier.
It was a secret project, and the incentives for most of the decision-makers involved were around producing “better-but-accountable” outcomes. There were multiple individuals to whom if you presented a sufficiently plausible argument, might have been able to stop the Manhattan project, up until the actual attack on Hiroshima. Fermi et al. had a pretty good chance of preventing the bomb from being used, and had other technical experts agreed that the risks were too high, I think the Manhattan project could have been stopped.
The current situation is much worse, with far more people having strong incentives to deceive others, themselves, and knowingly take risks.
All of that said, I still don’t think this means we have no choice. There are multiple ways of coordinating to solve a bad equilibrium, and I don’t think the incentives for pursuing AI are nearly so universal.
Most of the public recognizes that they have a lot of incentive to stop the development of ASI, even if right now it is more focused on job losses, they also recognize that the gains of AI are unlikely to be distributed evenly, and most Americans in particular care more about their chance of gaining in social status, than almost any other terminal outcome of policy (I can provide some citations if needed.)
More importantly, that public disapproval likely has a stable equilibrium in that they probably should be worried about the existential risk, whatever percentage it really is.
All that needs to follow is that the public disapproval needs to be strong enough and translate into a strong enough signal that policy makers are duly incentivize to take the concerns seriously, and policy make.
That isn’t super easy, but it isn’t pure fantasy at all. Everyone can see how the incentives for AI firms favor total race dynamics, but for most of humanity and even most Americans incentives are more than strong enough to prefer caution.