When we have something to publish we will do so. Generally our system cards and other publications contain evaluations of different aspects of safety and alignment by us and third parties. I expect that as capabilities grow and stakes of internal and external deployment are higher, we will continue and expand both our own evaluations and such collaborations.
(Not sure if you agree about this one?) You agree that if someone made an unaligned ASI while there is not also a comparably powerful aligned ASI, that would likely spell doom for humanity.
You agree that an unaligned ASI could emerge during research in general and in particular during large frontier training runs involving lots of compute.
You state that OpenAI is doing some activity related to checking whether or not that’s happening in their research.
(Not sure if you’re meaning to imply this or not?) You imply that those activities are somehow adequate to the task of preventing the creation of an unaligned ASI before the creation of an aligned ASI.
Do you agree with these bullet points? I would add this bullet point:
There’s no good public reason for anyone to believe that implication, whether in the form of a technical plan or analysis of why it’s feasible, or review from independent experts, or any discussion or debate on this point from you or anyone else at OpenAI.
I don’t really agree with any of these bulletpoints. I am not even sure we are on the same page of the definition of ASI and I don’t view “emerging” as a good way to describe training. I feel like we are getting into more fundamental disagreements which I covered to some extent here.
When we have something to publish we will do so. Generally our system cards and other publications contain evaluations of different aspects of safety and alignment by us and third parties. I expect that as capabilities grow and stakes of internal and external deployment are higher, we will continue and expand both our own evaluations and such collaborations.
So to recap,
(Not sure if you agree about this one?) You agree that if someone made an unaligned ASI while there is not also a comparably powerful aligned ASI, that would likely spell doom for humanity.
You agree that an unaligned ASI could emerge during research in general and in particular during large frontier training runs involving lots of compute.
You state that OpenAI is doing some activity related to checking whether or not that’s happening in their research.
(Not sure if you’re meaning to imply this or not?) You imply that those activities are somehow adequate to the task of preventing the creation of an unaligned ASI before the creation of an aligned ASI.
Do you agree with these bullet points? I would add this bullet point:
There’s no good public reason for anyone to believe that implication, whether in the form of a technical plan or analysis of why it’s feasible, or review from independent experts, or any discussion or debate on this point from you or anyone else at OpenAI.
And I wonder what you think of it.
I don’t really agree with any of these bulletpoints. I am not even sure we are on the same page of the definition of ASI and I don’t view “emerging” as a good way to describe training. I feel like we are getting into more fundamental disagreements which I covered to some extent here.