Hi Mikhail, thanks for offering your thoughts on this. I think having more public discussion on this is useful and I appreciate you taking the time to write this up.
I think your comment mostly applies to Guido in front of Anthropic, and not our hunger strike in front of Google DeepMind in London.
Hunger strikes can be incredibly powerful when there’s a just demand, a target who would either give in to the demand or be seen as a villain for not doing so, a wise strategy, and a group of supporters.
I don’t think these hunger strikes pass the bar. Their political demands are not what AI companies would realistically give in to because of a hunger strike by a small number of outsiders.
I don’t think I have been framing Demis Hassabis as a villain and if you think I did it would be helpful to add a source for why you believe this.
I’m asking Demis Hassabis to “publicly state that DeepMind will halt the development of frontier AI models if all the other major AI companies agree to do so.” which I think is a reasonable thing to state given all public statements he made regarding AI Safety. I think that is indeed something that a company such as Google DeepMind would give in.
A hunger strike can bring attention to how seriously you perceive an issue. If you know how to make it go viral, that is; in the US, hunger strikes are rarely widely covered by the media.
I’m currently in the UK, and I can tell you that there’s already been twopieces published on Business Insider. I’ve also given three interviews in the past 24 hours to journalists to contribute to major publications. I’ll try to add links later if / once these get published.
At the moment, these hunger strikes are people vibe-protesting. They feel like some awful people are going to kill everyone, they feel powerless, and so they find a way to do something that they perceive as having a chance of changing the situation.
Again, I’m pretty sure I haven’t framed people as “awful”, and would be great if you could provide sources to that statement. I also don’t feel powerless. My motivation for doing this was in part to provide support to Guido’s strike in front of Anthropic, which feels more like helping an ally, joining forces.
I find it actually empowering to be able to be completely honest about what I actually think DeepMind should do to help stop the AI race and receive so much support from all kinds of people on the street, including employees from Google, Google DeepMind, Meta and Sony. I am also grateful to have Denys with me, who flew from Amsterdam to join the hunger strike, and all the journalists who have taken the time to talk to us, both in person and remotely.
Action is better than inaction; but please stop and think of your theory of change for more than five minutes, if you’re planning to risk your life, and then don’t risk your life[1]; please pick actions thoughtfully and wisely and not because of the vibes[2].
I agree to the general point that taking decisions based on an actual theory of change is a much more effective way to have an impact in the world. I’ve personally thought quite a lot about why doing this hunger strike in front of DeepMind is net good, and I believe it’s having the intended impact, so I disagree with your implication that I’m basing my decisions on vibes. If you’d like to know more I’d be happy to talk to you in person in front of the DeepMind office or remotely.
Now, taking a step back and considering Guido’s strike, I want to say that even if you think that his actions were reckless and based on vibes, it’s worth evaluating whether his actions (and their consequences) will eventually turn out to be net negative. For one I don’t think I would have been out in front of DeepMind as I type this if it was not for Guido’s action, and I believe what we’re doing here in London is net good. But most importantly we’re still at the start of the strikes so it’s hard to tell what will happen as this continues. I’d be happy to have this discussion again at the end of the year, looking back.
Finally, I’d like to acknowledge the health risks involved. I’m personally looking over my health and there are some medics at King’s Cross that would be willing to help quickly if anything extreme was to happen. And given the length of the strikes so far I think what we’re doing is relatively safe, though I’m happy to be proven otherwise.
your comment mostly applies to Guido in front of Anthropic
Yep!
I don’t think I have been framing Demis Hassabis as a villain
A hunger strike is not a good tool if you don’t want to paint someone as a villain in the eyes of the public when they don’t give in to your demand.
publicly state that DeepMind will halt the development of frontier AI models if all the other major AI companies agree to do so.” which I think is a reasonable thing to state
It is vanishingly unlikely that all other major AI companies would agree to do so without the US government telling them to; this statement would be helpful, but only to communicate their position and not because of the commitment itself. Why not ask them to ask the government to stop everyone (maybe conditional on China agreeing to stop everyone in China)?
I’ve also given three interviews in the past 24 hours to journalists to contribute to major publications
If any of them go viral in the US with a good message, I’ll (somewhat) change my mind!
I disagree with your implication that I’m basing my decisions on vibes
This was mainly my impression after talking to Guido; but do you want to say more about the impact you think you’ll have?
I’d be happy to have this discussion again at the end of the year, looking back
(Can come back to it at the end of the year; if you have any advance predictions, they might be helpful to have posted!)
And given the length of the strikes so far I think what we’re doing is relatively safe, though I’m happy to be proven otherwise
I hope you remain safe and are not proven otherwise! Hunger strikes do carry negative risks though. Do you have particular plans for how long to be on the hunger strike for?
A hunger strike is not a good tool if you don’t want to paint someone as a villain in the eyes of the public when they don’t give in to your demand.
Is there any form of protest that doesn’t implicitly imply that the person you’re protesting is doing something wrong? When the thing wrong is “causing human extinction” it seems to me kind of hard for that to not automatically be assumed ‘villainous’.
(Asking genuinely, I think it quite probably the answer is ‘yes’.)
Something like: Hunger strikes are optimized hard specifically for painting someone as a villain because they decide to make someone suffer or die (or be inhumanely fed), this is different from other forms of protests that are more focused on, e.g., that specific decisions are bad and should be revoked, but don’t necessarily try to make people perceive the other side as evil.
I don’t really see the problem with painting people as evil in principle, given that some people are evil. You can argue against it in specific cases, but I think the case for AI CEOs being evil is strong enough that it can’t be dismissed out of hand.
The case in question is “AI CEOs are optimising for their short-term status/profits, and for believing things about the world which maximise their comfort, rather than doing the due diligence required of someone in their position, which is to seriously check whether their company is building something which kills everyone”
Whether this is a useful frame for one’s own thinking—or a good frame to deploy onto the public—I’m not fully sure, but I think it does need addressing. Of course it might also differ between CEOs. I think Demis and Dario are two of the CEOs who it’s relatively less likely to apply to, but also I don’t think it applies weakly enough for them to be dismissed out of hand even in their cases.
“People are on hunger strikes” is not really a lot of evidence for “AI CEOs are optimizing for their short-term status/profits and are not doing the due diligence” in the eyes of the public.
I don’t think there’s any problem with painting people and institutions as evil, I’m just not sure why you would want to do this here, as compared to other things, and would want people to have answers to how they imagine a hunger strike would paint AI companies/CEOs and what would be the impact of that, because I expect little that could move the needle.
It is vanishingly unlikely that all other major AI companies would agree to do so without the US government telling them to; this statement would be helpful, but only to communicate their position and not because of the commitment itself. Why not ask them to ask the government to stop everyone (maybe conditional on China agreeing to stop everyone in China)?
This seems to be exactly the point of the demand? This is a demand that would be cheap (perhaps even of negative cost) for DeepMind to accept (because the other AI companies wouldn’t agree to that), and would also be a major publicity win for the Pause AI crowd. Even counting myself skeptical of the hunger strikes, I think this is a very smart move.
the demand is that a specific company agrees to halt if everyone halts; this does not help in reality, because in fact it won’t be the case that everyone halts (abscent gov intervention).
Hi Mikhail, thanks for offering your thoughts on this. I think having more public discussion on this is useful and I appreciate you taking the time to write this up.
I think your comment mostly applies to Guido in front of Anthropic, and not our hunger strike in front of Google DeepMind in London.
I don’t think I have been framing Demis Hassabis as a villain and if you think I did it would be helpful to add a source for why you believe this.
I’m asking Demis Hassabis to “publicly state that DeepMind will halt the development of frontier AI models if all the other major AI companies agree to do so.” which I think is a reasonable thing to state given all public statements he made regarding AI Safety. I think that is indeed something that a company such as Google DeepMind would give in.
I’m currently in the UK, and I can tell you that there’s already been two pieces published on Business Insider. I’ve also given three interviews in the past 24 hours to journalists to contribute to major publications. I’ll try to add links later if / once these get published.
Again, I’m pretty sure I haven’t framed people as “awful”, and would be great if you could provide sources to that statement. I also don’t feel powerless. My motivation for doing this was in part to provide support to Guido’s strike in front of Anthropic, which feels more like helping an ally, joining forces.
I find it actually empowering to be able to be completely honest about what I actually think DeepMind should do to help stop the AI race and receive so much support from all kinds of people on the street, including employees from Google, Google DeepMind, Meta and Sony. I am also grateful to have Denys with me, who flew from Amsterdam to join the hunger strike, and all the journalists who have taken the time to talk to us, both in person and remotely.
I agree to the general point that taking decisions based on an actual theory of change is a much more effective way to have an impact in the world. I’ve personally thought quite a lot about why doing this hunger strike in front of DeepMind is net good, and I believe it’s having the intended impact, so I disagree with your implication that I’m basing my decisions on vibes. If you’d like to know more I’d be happy to talk to you in person in front of the DeepMind office or remotely.
Now, taking a step back and considering Guido’s strike, I want to say that even if you think that his actions were reckless and based on vibes, it’s worth evaluating whether his actions (and their consequences) will eventually turn out to be net negative. For one I don’t think I would have been out in front of DeepMind as I type this if it was not for Guido’s action, and I believe what we’re doing here in London is net good. But most importantly we’re still at the start of the strikes so it’s hard to tell what will happen as this continues. I’d be happy to have this discussion again at the end of the year, looking back.
Finally, I’d like to acknowledge the health risks involved. I’m personally looking over my health and there are some medics at King’s Cross that would be willing to help quickly if anything extreme was to happen. And given the length of the strikes so far I think what we’re doing is relatively safe, though I’m happy to be proven otherwise.
Thanks for responding!
Yep!
A hunger strike is not a good tool if you don’t want to paint someone as a villain in the eyes of the public when they don’t give in to your demand.
It is vanishingly unlikely that all other major AI companies would agree to do so without the US government telling them to; this statement would be helpful, but only to communicate their position and not because of the commitment itself. Why not ask them to ask the government to stop everyone (maybe conditional on China agreeing to stop everyone in China)?
If any of them go viral in the US with a good message, I’ll (somewhat) change my mind!
This was mainly my impression after talking to Guido; but do you want to say more about the impact you think you’ll have?
(Can come back to it at the end of the year; if you have any advance predictions, they might be helpful to have posted!)
I hope you remain safe and are not proven otherwise! Hunger strikes do carry negative risks though. Do you have particular plans for how long to be on the hunger strike for?
I have sent myself an email to arrive on December 20th to send you both a reminder about this thread.
Is there any form of protest that doesn’t implicitly imply that the person you’re protesting is doing something wrong? When the thing wrong is “causing human extinction” it seems to me kind of hard for that to not automatically be assumed ‘villainous’.
(Asking genuinely, I think it quite probably the answer is ‘yes’.)
Something like: Hunger strikes are optimized hard specifically for painting someone as a villain because they decide to make someone suffer or die (or be inhumanely fed), this is different from other forms of protests that are more focused on, e.g., that specific decisions are bad and should be revoked, but don’t necessarily try to make people perceive the other side as evil.
I don’t really see the problem with painting people as evil in principle, given that some people are evil. You can argue against it in specific cases, but I think the case for AI CEOs being evil is strong enough that it can’t be dismissed out of hand.
The case in question is “AI CEOs are optimising for their short-term status/profits, and for believing things about the world which maximise their comfort, rather than doing the due diligence required of someone in their position, which is to seriously check whether their company is building something which kills everyone”
Whether this is a useful frame for one’s own thinking—or a good frame to deploy onto the public—I’m not fully sure, but I think it does need addressing. Of course it might also differ between CEOs. I think Demis and Dario are two of the CEOs who it’s relatively less likely to apply to, but also I don’t think it applies weakly enough for them to be dismissed out of hand even in their cases.
“People are on hunger strikes” is not really a lot of evidence for “AI CEOs are optimizing for their short-term status/profits and are not doing the due diligence” in the eyes of the public.
I don’t think there’s any problem with painting people and institutions as evil, I’m just not sure why you would want to do this here, as compared to other things, and would want people to have answers to how they imagine a hunger strike would paint AI companies/CEOs and what would be the impact of that, because I expect little that could move the needle.
That is true. “People are on hunger strikes and the CEOs haven’t even commented” is (some) public evidence of “AI CEOs are unempathetic”
I misunderstood your point, I thought you were arguing against painting individuals as evil in general.
This seems to be exactly the point of the demand? This is a demand that would be cheap (perhaps even of negative cost) for DeepMind to accept (because the other AI companies wouldn’t agree to that), and would also be a major publicity win for the Pause AI crowd. Even counting myself skeptical of the hunger strikes, I think this is a very smart move.
the demand is that a specific company agrees to halt if everyone halts; this does not help in reality, because in fact it won’t be the case that everyone halts (abscent gov intervention).
I don’t think the point of hunger strikes is to achieve immediate material goals, but publicity/symbolic ones.