I think Jacob (OP) said “OpenAI is trying to directly build safe AGI.” and cited the charter and other statements as evidence of this claim. Then John replied that the charter and other statements are “not much evidence” either for or against this claim, because talk is cheap. I think that’s a reasonable point.
Separately, maybe John in fact believes that the charter and other statements are insincere lip service. If so, I would agree with you (Paul) that John’s belief is probably incorrect, based on my very limited knowledge. [Where I disagree with OpenAI, I presume that top leadership is acting sincerely to make a good future with safe AGI, but that they have mistaken beliefs about the hardness of alignment and other topics.]
I’d guess that is an overestimate of the number of people actually doing alignment research at OpenAI, as opposed to capabilities research in which people pay lip service to alignment. In particular, all of the RLHF work is basically capabilities work which makes alignment harder in the long term (because it directly selects for deception), while billing itself as “alignment”.
I think Jacob (OP) said “OpenAI is trying to directly build safe AGI.” and cited the charter and other statements as evidence of this claim. Then John replied that the charter and other statements are “not much evidence” either for or against this claim, because talk is cheap. I think that’s a reasonable point.
Separately, maybe John in fact believes that the charter and other statements are insincere lip service. If so, I would agree with you (Paul) that John’s belief is probably incorrect, based on my very limited knowledge. [Where I disagree with OpenAI, I presume that top leadership is acting sincerely to make a good future with safe AGI, but that they have mistaken beliefs about the hardness of alignment and other topics.]
I was replying to:
Thanks, sorry for misunderstanding.