Ah now I get what you were going for! Yes this is great! Definitely directionally what I am saying. Thank you for the interpretive effort.
(Also, again side note for others: I’m not only talking about alignment, value erosion between human-led AI-powered factions is also a concern, the whole framing is very abstract so very detail-agnostic)
Of course, I would say it goes even deeper than that, and my argument forces even more uncertainty on us: What you call progress in the “type 2 technologies” may be confounded by the type 1 progress. i.e, moral and coordination progress may just be a side effect of our economic progress, and therefore also might not mean that much either. It’s a huge historical question. I elaborate on this in the footnotes of the follow-up posta bit. e.g. democracy may be a side effect of human labor being temporarily valuable (Is democracy a fad?), modernization theory in academia, average income growth being possible and therefore capacity to care about others may be a side effect of slow human reproduction (Bostrom, Hanson). (Claude gives a long list of case studiesin academia to look at for this question).
In general, I feel like the importance of this issue (trying to disentangle how “real”, how much of a standalone force, moral and coordination progress is) has been underrated in EA[1] - and precisely for this reason: it makes our macro priors very different.
Anyway, unless you have a confident belief that all the type 2 progress definitely is not explained by type 1 progress (which seems hard to imagine), you should also reduce some of the update about the future you got from the type 2 progress—since it would happen when we approach AGI anyway. (Or I think in your framing, not evidence that alignment is any easier (?))
This may be a useful way to frame it, perhaps this macro question is the actual crux. Thank you.
Ah now I get what you were going for! Yes this is great! Definitely directionally what I am saying. Thank you for the interpretive effort.
(Also, again side note for others: I’m not only talking about alignment, value erosion between human-led AI-powered factions is also a concern, the whole framing is very abstract so very detail-agnostic)
Of course, I would say it goes even deeper than that, and my argument forces even more uncertainty on us: What you call progress in the “type 2 technologies” may be confounded by the type 1 progress. i.e, moral and coordination progress may just be a side effect of our economic progress, and therefore also might not mean that much either.
It’s a huge historical question. I elaborate on this in the footnotes of the follow-up post a bit.
e.g. democracy may be a side effect of human labor being temporarily valuable (Is democracy a fad?), modernization theory in academia, average income growth being possible and therefore capacity to care about others may be a side effect of slow human reproduction (Bostrom, Hanson). (Claude gives a long list of case studies in academia to look at for this question).
In general, I feel like the importance of this issue (trying to disentangle how “real”, how much of a standalone force, moral and coordination progress is) has been underrated in EA[1] - and precisely for this reason: it makes our macro priors very different.
Anyway, unless you have a confident belief that all the type 2 progress definitely is not explained by type 1 progress (which seems hard to imagine), you should also reduce some of the update about the future you got from the type 2 progress—since it would happen when we approach AGI anyway. (Or I think in your framing, not evidence that alignment is any easier (?))
This may be a useful way to frame it, perhaps this macro question is the actual crux. Thank you.
Perhaps because it’s a foundational assumption of our subculture/ideology that moral progress is a huge deal and that we can do more of it.