As an anecdote, a few months ago I met a former MIRI researcher while handing out flyers for PauseAI. We had a great conversation, and they were very concerned about x-risk.
When I asked if they would be willing to sign PauseAI US’s petition, they declined, stating that they don’t think a shutdown is feasible. I was very confused by this, because for those who are concerned about x-risk, I do not see a strong relationship between the feasibility of a shutdown and whether it is a good idea to advocate for one.
To expand on my point about asymmetry:
If the shutdown plan fails, then we are undertaking one of the other plans. If the other plans fail, we die (with unacceptably high probability). Accidentally getting a shutdown when you meant to proceed with caution is a win condition, at least temporarily, and it allows for many other plans to be improved and enacted. Accidentally proceeding with caution when you meant to get a shutdown is walking on thin an ice. The distance from shutdown to death is larger than the distance from proceeding with caution to death.
To address another viewpoint: If you are concerned about x-risk and you believe that all effort going toward advocating for a shutdown is wasted, and that the world would be better off if no one talked about a shutdown, I think you’re simply confused about normal social and political dynamics.
As an anecdote, a few months ago I met a former MIRI researcher while handing out flyers for PauseAI. We had a great conversation, and they were very concerned about x-risk.
When I asked if they would be willing to sign PauseAI US’s petition, they declined, stating that they don’t think a shutdown is feasible. I was very confused by this, because for those who are concerned about x-risk, I do not see a strong relationship between the feasibility of a shutdown and whether it is a good idea to advocate for one.
To expand on my point about asymmetry: If the shutdown plan fails, then we are undertaking one of the other plans. If the other plans fail, we die (with unacceptably high probability). Accidentally getting a shutdown when you meant to proceed with caution is a win condition, at least temporarily, and it allows for many other plans to be improved and enacted. Accidentally proceeding with caution when you meant to get a shutdown is walking on thin an ice. The distance from shutdown to death is larger than the distance from proceeding with caution to death.
To address another viewpoint: If you are concerned about x-risk and you believe that all effort going toward advocating for a shutdown is wasted, and that the world would be better off if no one talked about a shutdown, I think you’re simply confused about normal social and political dynamics.