I like your analysis. I haven’t thought deeply about the particulars, but I agree that we should be able to observe evidence one way or the other right now. I’ve just found it prohibitively difficult (probably due to my own lack of skill) to encourage honestly looking at the present evidence in this particular case. So I was hoping that we could set some predictions and then revisit them, to bypass the metacognitive blindspot effect.
(And in terms of numbers, I would guess that the emotional cluster are a majority of people involved.)
That’s my gut impression too. And part of what I care about here is, if the proportion of such people is large enough, the social dynamics of what seems salient will be shaped by these emotional mechanisms, and that effect will masquerade as objectivity. Social impacts on epistemology are way stronger than I think most people realize or account for.
Suppose there is a consensus belief, and suppose that it’s totally correct. If funders, and more generally anyone who can make stuff happen (e.g. builders and thinkers), use this totally correct consensus belief to make local decisions about where to allocate resources, and they don’t check the global margin, then they will in aggregrate follow a portfolio of strategies that is incorrect. The make-stuff-happeners will each make happen the top few things on their list, and leave the rest undone. The top few things will be what the consensus says is most important——in our case, projects that help if AGI comes within 10 years. If a project helps in 30 years, but not 10 years, then it doesn’t get any funding at all. This is not the right global portfolio; it oversaturates fast interventions and leaves slow interventions undone.
I like your analysis. I haven’t thought deeply about the particulars, but I agree that we should be able to observe evidence one way or the other right now. I’ve just found it prohibitively difficult (probably due to my own lack of skill) to encourage honestly looking at the present evidence in this particular case. So I was hoping that we could set some predictions and then revisit them, to bypass the metacognitive blindspot effect.
That’s my gut impression too. And part of what I care about here is, if the proportion of such people is large enough, the social dynamics of what seems salient will be shaped by these emotional mechanisms, and that effect will masquerade as objectivity. Social impacts on epistemology are way stronger than I think most people realize or account for.
Obvious enough but worth saying explicitly: It’s not just impacts on epistemology, but also collective behavior. From https://www.lesswrong.com/posts/sTDfraZab47KiRMmT/views-on-when-agi-comes-and-on-strategy-to-reduce#My_views_on_strategy :