I think the framing “alignment research is preparadigmatic” might be heavily misunderstood. The term “preparadigmatic” of course comes from Thomas Kuhn’s The Structure of Scientific Revolutions. My reading of this says that a paradigm is basically an approach to solving problems which has been proven to work, and that the correct goal of preparadigmatic research should be to do research generally recognized as impressive.
For example, Kuhn says in chapter 2 that “Paradigms gain their status because they are more successful than their competitors in solving a few problems that the group of practitioners has come to recognize as acute.” That is, lots of researchers have different ontologies/approaches, and paradigms are the approaches that solve problems that everyone, including people with different approaches, agrees to be important. This suggests that to the extent alignment is still preparadigmatic, we should try to solve problems recognized as important by, say, people in each of the five clusters of alignment researchers (e.g. Nate Soares, Dan Hendrycks, Paul Christiano, Jan Leike, David Bau).
I think this gets twisted in some popular writings on LessWrong. John Wentworth writes that a researcher in a preparadigmatic field should spend lots of time explaining their approaches:
Because the field does not already have a set of shared frames—i.e. a paradigm—you will need to spend a lot of effort explaining your frames, tools, agenda, and strategy. For the field, such discussion is a necessary step to spreading ideas and eventually creating a paradigm.
I think this is misguided. A paradigm is not established by ideas diffusing between researchers with different frames until they all agree on some weighted average of the frames. A paradigm is established by research generally recognized as impressive, which proves the correctness of (some aspect of) someone’s frames. So rather than trying to communicate one’s frame to everyone, one should communicate with other researchers to get an accurate sense of what problems they think are important, and then work on those problems using one’s own frames. (edit: of course, before this is possible one should develop one’s frames to solve some problems)
I think the framing “alignment research is preparadigmatic” might be heavily misunderstood. The term “preparadigmatic” of course comes from Thomas Kuhn’s The Structure of Scientific Revolutions. My reading of this says that a paradigm is basically an approach to solving problems which has been proven to work, and that the correct goal of preparadigmatic research should be to do research generally recognized as impressive.
For example, Kuhn says in chapter 2 that “Paradigms gain their status because they are more successful than their competitors in solving a few problems that the group of practitioners has come to recognize as acute.” That is, lots of researchers have different ontologies/approaches, and paradigms are the approaches that solve problems that everyone, including people with different approaches, agrees to be important. This suggests that to the extent alignment is still preparadigmatic, we should try to solve problems recognized as important by, say, people in each of the five clusters of alignment researchers (e.g. Nate Soares, Dan Hendrycks, Paul Christiano, Jan Leike, David Bau).
I think this gets twisted in some popular writings on LessWrong. John Wentworth writes that a researcher in a preparadigmatic field should spend lots of time explaining their approaches:
I think this is misguided. A paradigm is not established by ideas diffusing between researchers with different frames until they all agree on some weighted average of the frames. A paradigm is established by research generally recognized as impressive, which proves the correctness of (some aspect of) someone’s frames. So rather than trying to communicate one’s frame to everyone, one should communicate with other researchers to get an accurate sense of what problems they think are important, and then work on those problems using one’s own frames. (edit: of course, before this is possible one should develop one’s frames to solve some problems)
Strong agree. 👍