I wrote that comment before reading the clarification about there being an intended context of a time horizon. I agree that it’d take longer to develop FAI than to develop an UFAI, all else equal. (Of course, all else is not equal, and you did write about intelligence threshold and not time.)
I think there is significant speedup in the time it took us to build space shuttles gained through outliers in intelligence, compared to the time it would take to do that for a species of uniform ability that was only barely smart enough to invent windmills, but I expect they’d get there, barring defeaters (to use Chalmers’ term), if they manage to stumble upon a science-generating social institution, and avoid collapse of their civilization for long enough to accumulate the necessary knowledge. People that are not that smart can do science, and learn science, as they can learn language or any other skill, it just takes them much longer to learn it or find useful abstract insights, and each individual can master less. We (as a civilization) don’t use people of low ability for this purpose only because there is a much more efficient alternative.
I wrote that comment before reading the clarification about there being an intended context of a time horizon. I agree that it’d take longer to develop FAI than to develop an UFAI, all else equal. (Of course, all else is not equal, and you did write about intelligence threshold and not time.)
I think there is significant speedup in the time it took us to build space shuttles gained through outliers in intelligence, compared to the time it would take to do that for a species of uniform ability that was only barely smart enough to invent windmills, but I expect they’d get there, barring defeaters (to use Chalmers’ term), if they manage to stumble upon a science-generating social institution, and avoid collapse of their civilization for long enough to accumulate the necessary knowledge. People that are not that smart can do science, and learn science, as they can learn language or any other skill, it just takes them much longer to learn it or find useful abstract insights, and each individual can master less. We (as a civilization) don’t use people of low ability for this purpose only because there is a much more efficient alternative.