Thank you for also pointing at CESIA and the website. I checked the website and have some feedback.
PLEASE CHANGE IT! I strongly suggest adding x-risk or loss of control back to the website.
Right now if a staffer or policymaker goes to the website they don’t find x-risk and loss of control. I think policymakers we talk to visiting the website might be actively bad. And I would suggest not just adding loss of control but making it clear that you are talking about extinction risk.
Perspective of staffer: “If X-risk was real, then CESIA would have obviously mentioned it on their website. Everything else would be insane. I was worried there for a bit, but I guess these guys at PAI and CAI are crazy.”
You could even link to more extensive things add a whole tab mentioning gradual disempowerment, loss of control, x-risk, concentration of power, geo-political risks … .
Minimum E.g. on the frontpage, the bottom and in the about section there is this paragraph:
Our mission is to prevent and mitigate major AI risks by informing policymakers, raising public awareness, and conducting policy research.
changing it to something like: “Our mission is to prevent and mitigate major AI risks including risk of loss of control and risk of human extinction by informing policymakers, raising public awareness, and conducting policy research.”
> Our consultation response advocates for clear, internationally enforced AI red lines to prevent unacceptable risks and misuses, including those related to autonomous weapons of mass destruction, biological risks, offensive cyber capabilities, harmful manipulation, and child safety.
Thank you for submitting something and again mentioning the deficit in your post and again mention x-risk, loss of control, gradual disempowerment.
Even Anthropic on their website mentions loss of control in their policy priorities: > But transparency alone is not sufficient to safeguard against the most serious risks posed by powerful AI, including the ability to help create biological weapons or carry out cyber operations and the loss of control of AI systems. Our Advanced AI Framework lays out what we think governments should do about these risks in the near term.
I put an appointment in my calendar to check the website again next week.
---
PS: An exercise I suggest to anyone who feels in any way weird talking about x-risk: Talk to ten random people and say: “Sorry, do you have a moment to chat about something? Did you know AI might kill everyone in a few years?” (not because this is an effective way to engage but to get it out of your system). Repeat until the part of saying AI might kill everyone doesn’t feel weird anymore.
“PLEASE CHANGE IT!” The vast majority of CeSIA’s impact (99%) is not via our website but through the rest of our engagement, so I don’t consider this a priority, even though putting the FAQ on AI risks back on our backlog will be done at some point. The role of the website is mostly to open doors.
How do you know? Even when the intended impact is not via the website it’s still possible that the website has more impact that you intend to and that impact is negative.
Hi Charbel, Sorry, looking back now at my comment it’s pretty weird and too strongly worded. Especially asking you to change it right now and checking back. Thank you, for your very diplomatic answer! Maybe to rephrase it how I would write it now: I agree that it would be good to have the Q&A back on the website. I think the “invisible cost” of not having “Risk of loss of control” present is often underestimated. I am not saying that as useful assistant one can’t help with the topic of the day without mentioning x-risk, but that the cost of systematically keeping it out of sight is often underestimated. And I have seen people who have been in the community for years, feel awkward talking directly about the topic. Best regards, Benjamin
Lots of helpful points.
Thank you for also pointing at CESIA and the website. I checked the website and have some feedback.
PLEASE CHANGE IT! I strongly suggest adding x-risk or loss of control back to the website.
Right now if a staffer or policymaker goes to the website they don’t find x-risk and loss of control. I think policymakers we talk to visiting the website might be actively bad. And I would suggest not just adding loss of control but making it clear that you are talking about extinction risk.
Perspective of staffer: “If X-risk was real, then CESIA would have obviously mentioned it on their website. Everything else would be insane. I was worried there for a bit, but I guess these guys at PAI and CAI are crazy.”
You could even link to more extensive things add a whole tab mentioning gradual disempowerment, loss of control, x-risk, concentration of power, geo-political risks … .
Minimum E.g. on the frontpage, the bottom and in the about section there is this paragraph:
changing it to something like: “Our mission is to prevent and mitigate major AI risks including risk of loss of control and risk of human extinction by informing policymakers, raising public awareness, and conducting policy research.”
Also looking at https://cesia.org/en/publications/les-contributions-du-cesia-au-dialogue-mondial-des-nations-unies-sur-la-gouvernance-de-lia/:
> Our consultation response advocates for clear, internationally enforced AI red lines to prevent unacceptable risks and misuses, including those related to autonomous weapons of mass destruction, biological risks, offensive cyber capabilities, harmful manipulation, and child safety.
Thank you for submitting something and again mentioning the deficit in your post and again mention x-risk, loss of control, gradual disempowerment.
Even Anthropic on their website mentions loss of control in their policy priorities:
> But transparency alone is not sufficient to safeguard against the most serious risks posed by powerful AI, including the ability to help create biological weapons or carry out cyber operations and the loss of control of AI systems. Our Advanced AI Framework lays out what we think governments should do about these risks in the near term.
I put an appointment in my calendar to check the website again next week.
---
PS: An exercise I suggest to anyone who feels in any way weird talking about x-risk: Talk to ten random people and say: “Sorry, do you have a moment to chat about something? Did you know AI might kill everyone in a few years?” (not because this is an effective way to engage but to get it out of your system). Repeat until the part of saying AI might kill everyone doesn’t feel weird anymore.
Thanks Benjamin.
“PLEASE CHANGE IT!” The vast majority of CeSIA’s impact (99%) is not via our website but through the rest of our engagement, so I don’t consider this a priority, even though putting the FAQ on AI risks back on our backlog will be done at some point. The role of the website is mostly to open doors.
But thanks for the feedback.
How do you know? Even when the intended impact is not via the website it’s still possible that the website has more impact that you intend to and that impact is negative.
number of visitors on our website vs number of views on videos such as https://www.youtube.com/watch?v=ZP7T6WAK3Ow
Hi Charbel,
Sorry, looking back now at my comment it’s pretty weird and too strongly worded. Especially asking you to change it right now and checking back. Thank you, for your very diplomatic answer!
Maybe to rephrase it how I would write it now: I agree that it would be good to have the Q&A back on the website. I think the “invisible cost” of not having “Risk of loss of control” present is often underestimated. I am not saying that as useful assistant one can’t help with the topic of the day without mentioning x-risk, but that the cost of systematically keeping it out of sight is often underestimated. And I have seen people who have been in the community for years, feel awkward talking directly about the topic.
Best regards,
Benjamin