IMO this discussion suffers from lumping together all academic disciplines, whereas the actual situation in different disciplines is very different. In particular, “hard” disciplines such as mathematics and physics have a very impressive track record and I don’t see a reason to believe in any decline in recent decades. On the other hand, it doesn’t seem to be the case at all for academic philosophy (which is not to say that no individual philosophers have produced important ideas). So, it shouldn’t come as a surprise that interesting philosophical ideas such as updateless/anthropic decision theory are slow to get traction in academic philosophy.
It is also not a major surprise that so few computer scientists are working on AI alignment. This is not because academic computer science is fundamentally broken, it’s just because the best case scenario for almost all scientists is choosing problems to work on because they seem interesting, not because they are important in terms of consequences in the real world (other sources of motivations include gaining prestige and receiving grants). In order to get computer scientists interested in AI alignment, the alignment community has to produce a critical mass of impressive mathematical results that seem interesting regardless of their applications. Unfortunately, we are not quite there yet. For example, UDT is not sufficiently appetizing since it still lacks solid mathematical foundation. (Incidentally I also think that the case for the AI alignment importance of UDT is only marginally strong and we should work more on directions closer to the computational learning theory mainstream.)
Historically, I think that major conceptual breakthroughs were made by a single person (or maybe a small group) taking a bold risk which paid off. Once a core of tangible results is established, the rest of the academia dives in to pick the low-hanging fruit and build up further. This is the bottleneck we need to push through in order for AI alignment to gain traction.
I would like to add to Vanessa Kowalski, that it would be useful not only to talk about academic disciplines separately, but also to look at academia of different countries separately. Are y’all talking about academia in the US or in the whole world? I suspect the former. Is it like that in Europe too? What about China? Australia? Japan? India? Russia?
IMO this discussion suffers from lumping together all academic disciplines, whereas the actual situation in different disciplines is very different. In particular, “hard” disciplines such as mathematics and physics have a very impressive track record and I don’t see a reason to believe in any decline in recent decades. On the other hand, it doesn’t seem to be the case at all for academic philosophy (which is not to say that no individual philosophers have produced important ideas). So, it shouldn’t come as a surprise that interesting philosophical ideas such as updateless/anthropic decision theory are slow to get traction in academic philosophy.
It is also not a major surprise that so few computer scientists are working on AI alignment. This is not because academic computer science is fundamentally broken, it’s just because the best case scenario for almost all scientists is choosing problems to work on because they seem interesting, not because they are important in terms of consequences in the real world (other sources of motivations include gaining prestige and receiving grants). In order to get computer scientists interested in AI alignment, the alignment community has to produce a critical mass of impressive mathematical results that seem interesting regardless of their applications. Unfortunately, we are not quite there yet. For example, UDT is not sufficiently appetizing since it still lacks solid mathematical foundation. (Incidentally I also think that the case for the AI alignment importance of UDT is only marginally strong and we should work more on directions closer to the computational learning theory mainstream.)
Historically, I think that major conceptual breakthroughs were made by a single person (or maybe a small group) taking a bold risk which paid off. Once a core of tangible results is established, the rest of the academia dives in to pick the low-hanging fruit and build up further. This is the bottleneck we need to push through in order for AI alignment to gain traction.
I would like to add to Vanessa Kowalski, that it would be useful not only to talk about academic disciplines separately, but also to look at academia of different countries separately. Are y’all talking about academia in the US or in the whole world? I suspect the former. Is it like that in Europe too? What about China? Australia? Japan? India? Russia?