That post seems to mainly address high P(doom) arguments and reject them. I agree with some of those arguments and the rejection of high P(doom). I don’t see as direct of a relevance to my previous comment. As for the broader point of self-selection, I think this is important, but cuts both ways: funders are selected to be competent generalists (and are biased towards economic arguments) as such they are pre-disposed to under-update on inside views. As an extreme case of this consider e.g. Bryan Caplan.
Here are comments on two of Nuno’s arguments which do apply to AGI timelines:
(A) “Difference between in-argument reasoning and all-things-considered reasoning” this seems closest to my point (1) which is often an argument for shorter timelines.
(B) “there is a small but intelligent community of people who have spent significant time producing some convincing arguments about AGI, but no community which has spent the same amount of effort”. This strikes me as important, but likely not true without heavy caveats. Academia celebrates works pointing out clear limitations of existing work e.g. Will Merill’s work [1,2] and Inverse Scaling Laws. It’s true that there’s no community organized around this work, but the important variables are incentives/scale/number-of-researcher-hours—not community.
That post seems to mainly address high P(doom) arguments and reject them. I agree with some of those arguments and the rejection of high P(doom). I don’t see as direct of a relevance to my previous comment. As for the broader point of self-selection, I think this is important, but cuts both ways: funders are selected to be competent generalists (and are biased towards economic arguments) as such they are pre-disposed to under-update on inside views. As an extreme case of this consider e.g. Bryan Caplan.
Here are comments on two of Nuno’s arguments which do apply to AGI timelines:
(A) “Difference between in-argument reasoning and all-things-considered reasoning” this seems closest to my point (1) which is often an argument for shorter timelines.
(B) “there is a small but intelligent community of people who have spent significant time producing some convincing arguments about AGI, but no community which has spent the same amount of effort”. This strikes me as important, but likely not true without heavy caveats. Academia celebrates works pointing out clear limitations of existing work e.g. Will Merill’s work [1,2] and Inverse Scaling Laws. It’s true that there’s no community organized around this work, but the important variables are incentives/scale/number-of-researcher-hours—not community.
Yeah, I agree that the disagreement is probably more important to resolve, and I haven’t much addressed that.