My guess is that the most likely result would be that (1) by no means all AI researchers would take the pledge (because such things never get universally adopted), and then (2) the government would preferentially hire people who hadn’t taken it or who would credibly undertake to break it in the name of National Security, and then (3) the first-order effect of the pledge would be to reduce the fraction of people in any government AI effort who are at all concerned about Friendliness.
That doesn’t seem to be a win.
I suppose that if you could get it near-universally adopted it might work, but that would require a degree of buy-in I see no way of getting.
Yes, this is the key question. But if we grant that current AI work is not likely to lead to AGI, and that a new generation of research and researchers is needed, then perhaps this sort of pledge can be the norm.
It doesn’t need to be a formal pledge, just a well-known point of view. If researchers are known to be part of the MIRI circle, that is already halfway to rendering them “suspect” of having a strong safety concern.
My guess is that the most likely result would be that (1) by no means all AI researchers would take the pledge (because such things never get universally adopted), and then (2) the government would preferentially hire people who hadn’t taken it or who would credibly undertake to break it in the name of National Security, and then (3) the first-order effect of the pledge would be to reduce the fraction of people in any government AI effort who are at all concerned about Friendliness.
That doesn’t seem to be a win.
I suppose that if you could get it near-universally adopted it might work, but that would require a degree of buy-in I see no way of getting.
Yes, this is the key question. But if we grant that current AI work is not likely to lead to AGI, and that a new generation of research and researchers is needed, then perhaps this sort of pledge can be the norm.
It doesn’t need to be a formal pledge, just a well-known point of view. If researchers are known to be part of the MIRI circle, that is already halfway to rendering them “suspect” of having a strong safety concern.