Thinking more about this, maybe, the most important critique of PauseAI is not their epistemology; it’s that, empirically, they have a low reputation: the FLI pushing the superintelligence statement has garnered a much bigger support—while PauseAI, with a similar statement, didn’t get the support.
Another argument: the School for Moral Ambition (100k views on this video), or ControlAI (100k subscribers to their newsletter if I remember), seems to be able to get a lot of public support as well, and this might be more effective at getting more people to sign on to something?
So this is a weird phenomenon: People agree that we need to slow down or pause (here’s a survey we conducted at CeSIA to demonstrate this), but it seems they don’t want to associate with PauseAI?
Or maybe there is an implicit rule that advocacy organizations are lower-status than research organizations, and not asking for what you ultimately want in the title of your organizations is what wins in the end?
If you are thinking in term of raw number of signatures, I think you are conflating low reach with low reputation. We do have lower reach than FLI, but we are not operating with anywhere near the same resources. FLI 2024 total expenditures were $17M.
If you are thinking in terms of number of big names, there is also a question of reach. Our network is not as developed, again coming down to a matter of available resources. But there is also a selection effect. People like Bengio deliberately keep some distance from PauseAI advocacy to protect a middle-ground position, which suppresses some of our visible support. I guess you could call that advocacy being lower-status than research. That being said, I think it’s very important for us to maintain a good reputation and we continue working on it.
On School for Moral Ambition / ControlAI: I think these are doing a different work than us, and shallower—and I don’t mean shallow in a pejorative sense here but descriptive: getting a lot of people to sign a statement or to subscribe to your newsletter is the visible/top layer. What we are building is the thing underneath, the deep organised buy-in, individuals who start owning the issue and moving their social network. It’s also worth noting again that Bregman started with a large pre-existing following, and that ControlAI’s operating budget is an order of magnitude above ours. We see ourselves as complementary with these organisations.
And to bounce on your CeSIA survey: to me it shows that the latent constituency is already here, so the next big move is to organise it.
Thinking more about this, maybe, the most important critique of PauseAI is not their epistemology; it’s that, empirically, they have a low reputation: the FLI pushing the superintelligence statement has garnered a much bigger support—while PauseAI, with a similar statement, didn’t get the support.
Another argument: the School for Moral Ambition (100k views on this video), or ControlAI (100k subscribers to their newsletter if I remember), seems to be able to get a lot of public support as well, and this might be more effective at getting more people to sign on to something?
So this is a weird phenomenon: People agree that we need to slow down or pause (here’s a survey we conducted at CeSIA to demonstrate this), but it seems they don’t want to associate with PauseAI?
Or maybe there is an implicit rule that advocacy organizations are lower-status than research organizations, and not asking for what you ultimately want in the title of your organizations is what wins in the end?
Fascinating
Thanks for the comment. Let me try to unpack.
On the statement comparison:
If you are thinking in term of raw number of signatures, I think you are conflating low reach with low reputation. We do have lower reach than FLI, but we are not operating with anywhere near the same resources. FLI 2024 total expenditures were $17M.
If you are thinking in terms of number of big names, there is also a question of reach. Our network is not as developed, again coming down to a matter of available resources. But there is also a selection effect. People like Bengio deliberately keep some distance from PauseAI advocacy to protect a middle-ground position, which suppresses some of our visible support. I guess you could call that advocacy being lower-status than research. That being said, I think it’s very important for us to maintain a good reputation and we continue working on it.
On School for Moral Ambition / ControlAI: I think these are doing a different work than us, and shallower—and I don’t mean shallow in a pejorative sense here but descriptive: getting a lot of people to sign a statement or to subscribe to your newsletter is the visible/top layer. What we are building is the thing underneath, the deep organised buy-in, individuals who start owning the issue and moving their social network.
It’s also worth noting again that Bregman started with a large pre-existing following, and that ControlAI’s operating budget is an order of magnitude above ours. We see ourselves as complementary with these organisations.
And to bounce on your CeSIA survey: to me it shows that the latent constituency is already here, so the next big move is to organise it.