I will answer now, Matilda/Maxime probably went to bed already.
“The right terminal goal is a binding international pause treaty with verification/enforcement” --> I agree functionally, ideally a pause for at least a couple of years, or at minimum the capacity to pause. Though I think there are much more palatable ways to present this to policymakers than “pause.
Could you expand upon this? I am partially unsure what you mean. Also interested in which ways you would present it to policymakers? (Red lines?) I think there are also good reasons to pause earlier because later on a Pause might be more difficult/impossible to implement + gradual disempowerment + …
> Policymakers need strong enough incentives to act I think this depends. 1)The current administration in the US is quite weird. 2) I think for anyone outside of the admin to ask for a ban of Fable specifically (not that we would have wanted to) we would have needed strong incentives for policymakers to act. The more ASI/AGI pilled policymakers are the less incentives they need and maybe things align well in other ways. Also Section II of the post. A side thought: WuWei don’t try to fight, try to work with the system and create a system which already does what you want. Europe is also waking up to the security and sovereignty implications of strong AI systems. Also, the better the incentives (for a good version of a pause) the better the version of a pause we will be able to implement.
> “The movement can grow fast without breaking”—I don’t know
Do you have specific scenarios in mind? The federation model is designed to help stop it but ofc. nothing is foolproof. We try to be positive EV compared to the alternative like you mention next. More money would help as well!
> it seems to me that whoever is in contact with the White House in the community has much more leverage
If PauseAI grew in a timeline with high AI salience this might shift, though the current White House is weird.
> not that important (people in administrations) Mostly agree, adding some thoughts. I think a big enough public movement will also influence these people but it’s very different from policymakers. This is where we need good think tanks/policy people. Because the situation emerged we also did a lot of talking to policymakers in Germany and even the bureaucracy once. Another way to influence the bureaucracy is from the outside when policy interacts with bureaucracy e.g. right now the new German AI security institute is being created and can be shaped to be more useful for a Pause than e.g. the UK one.
I will answer now, Matilda/Maxime probably went to bed already.
Could you expand upon this? I am partially unsure what you mean. Also interested in which ways you would present it to policymakers? (Red lines?)
I think there are also good reasons to pause earlier because later on a Pause might be more difficult/impossible to implement + gradual disempowerment + …
> Policymakers need strong enough incentives to act
I think this depends. 1)The current administration in the US is quite weird. 2) I think for anyone outside of the admin to ask for a ban of Fable specifically (not that we would have wanted to) we would have needed strong incentives for policymakers to act. The more ASI/AGI pilled policymakers are the less incentives they need and maybe things align well in other ways. Also Section II of the post.
A side thought: WuWei don’t try to fight, try to work with the system and create a system which already does what you want. Europe is also waking up to the security and sovereignty implications of strong AI systems.
Also, the better the incentives (for a good version of a pause) the better the version of a pause we will be able to implement.
> “The movement can grow fast without breaking”—I don’t know
Do you have specific scenarios in mind? The federation model is designed to help stop it but ofc. nothing is foolproof. We try to be positive EV compared to the alternative like you mention next.
More money would help as well!
> it seems to me that whoever is in contact with the White House in the community has much more leverage
If PauseAI grew in a timeline with high AI salience this might shift, though the current White House is weird.
> not that important (people in administrations)
Mostly agree, adding some thoughts. I think a big enough public movement will also influence these people but it’s very different from policymakers. This is where we need good think tanks/policy people. Because the situation emerged we also did a lot of talking to policymakers in Germany and even the bureaucracy once.
Another way to influence the bureaucracy is from the outside when policy interacts with bureaucracy e.g. right now the new German AI security institute is being created and can be shaped to be more useful for a Pause than e.g. the UK one.