How many “important” things do you expect the entire field of alignment to achieve in the next 4 years? All of the questions above are highly sensitive to this number.
It’s really hard to say because the alignment field might grow exponentially. I have the impression that this is already happening, with e.g. SERI MATS recently having started a whole bunch of new alignment researchers? So we should probably expect a lot of growth.
One thing I’m unsure about is the extent to which this growth all funnels into the existing research programs vs creates new research programs. Insofar as everyone creates new research programs, it seems like the answer should still be predictable when averaging over researchers.
So, likely do I expect a skilled researcher on a promising-ish research program to be to come up with a big finding in four years? Probably the smartest place would be to go with base rates; what have we seen so far in alignment research. But I don’t feel strong enough in history to list the major programs and achievements.
I tried thinking about each of the sub-directions I could think of in the above 4 research programs, and considered my gut feeling for the probability of whether the researchers would find something in those sub-directions, and added them up. I got the result of 2.2 expected findings. Obviously this should be taken with a huge grain of salt.
To help clarify, if anyone wants to present a historical research result that they would like to know whether I would consider an achievement, then I would be open to rating it.
How many “important” things do you expect the entire field of alignment to achieve in the next 4 years? All of the questions above are highly sensitive to this number.
It’s really hard to say because the alignment field might grow exponentially. I have the impression that this is already happening, with e.g. SERI MATS recently having started a whole bunch of new alignment researchers? So we should probably expect a lot of growth.
One thing I’m unsure about is the extent to which this growth all funnels into the existing research programs vs creates new research programs. Insofar as everyone creates new research programs, it seems like the answer should still be predictable when averaging over researchers.
So, likely do I expect a skilled researcher on a promising-ish research program to be to come up with a big finding in four years? Probably the smartest place would be to go with base rates; what have we seen so far in alignment research. But I don’t feel strong enough in history to list the major programs and achievements.
I tried thinking about each of the sub-directions I could think of in the above 4 research programs, and considered my gut feeling for the probability of whether the researchers would find something in those sub-directions, and added them up. I got the result of 2.2 expected findings. Obviously this should be taken with a huge grain of salt.
To help clarify, if anyone wants to present a historical research result that they would like to know whether I would consider an achievement, then I would be open to rating it.
You might also be interested in this comment, which lists some things I consider to be achievement and non-achievements: https://manifold.markets/tailcalled/will-tailcalled-think-that-the-infr#DzXOJ7Tw4EURocWdzjlX