Yes, but the actual power didn’t come from the architectural difference, even if the architectural difference got us the ability to process culture. The actual power came from the culture. That’s my point.
ETA: Are you saying that AI could discontinuously develop culture much more quickly than humans?
I guess what I was trying to say is (although I think I’ve partially figured out what you meant; see next paragraph), cultural evolution is a process that acquires adaptations slowly-ish and transmits previously-acquired adaptations to new organisms quickly, while biological evolution is a process that acquires adaptations very slowly and transmits previously-acquired adaptations to new organisms quickly. You seem to be comparing the rate at which cultural evolution acquires adaptations to the rate at which biological evolution transmits previously-acquired adaptations to new organisms, and concluding that cultural evolution is slower.
Re-reading the part of your post where you talked about AI takeoff speeds, you argue (which I hadn’t understood before) that the rise of humans was fast on an evolutionary timescale, and slow on a cultural timescale, so that if it was due to an evolutionary change, it must involve a small change that had a large effect on capabilities, so that a large change will occur very suddenly if we mimic evolution quickly, while if it was due to a cultural change, it was probably a large change, so mimicking culture quickly won’t produce a large effect on capabilities unless it is extremely quick.
This clarifies things, but I don’t agree with the claim. I think slow changes in the intelligence of a species is compatible with fast changes in its capabilities even if the changes are mainly in raw innovative ability rather than cultural learning. Innovations can increase ability to innovate, causing a positive feedback loop. A species could have high enough cultural learning ability for innovations to be transmitted over many generations without having the innovative ability to ever get the innovations that will kick off this loop. Then, when they start slowing gaining innovative ability, the innovations accumulated into cultural knowledge gradually increase, until they reach the feedback loop and the rate of innovation becomes more determined by changes in pre-existing innovations than by changes in raw innovative ability. There doesn’t even have to be any evolutionary changes in the period in which innovation rate starts to get dramatic.
If you don’t buy this story, then it’s not clear why the changes being in cultural learning ability rather than in raw innovative ability would remove the need for a discontinuity. After all, our cultural learning ability went from not giving us much advantage over other animals to “accumulating decisive technological dominance in an evolutionary eyeblink” in an evolutionary eyeblink (quotation marks added for ease of parsing). Does this mean our ability to learn from culture must have greatly increased from a small change? You argue in the post that there’s no clear candidate for what such a discontinuity in cultural learning ability could look like, but this seems just as true to me for raw innovative ability.
Perhaps you could argue that it doesn’t matter if there’s a sharp discontinuity in cultural learning ability because you can’t learn from a culture faster than the culture learns things to teach you. In this case, yes, perhaps I would say that AI-driven culture could make advancements that look discontinuous on a human scale. Though I’m not entirely sure what that would look like, and I admit it does sound kind of soft-takeoffy.
Yes, but the actual power didn’t come from the architectural difference, even if the architectural difference got us the ability to process culture. The actual power came from the culture. That’s my point.
ETA: Are you saying that AI could discontinuously develop culture much more quickly than humans?
I guess what I was trying to say is (although I think I’ve partially figured out what you meant; see next paragraph), cultural evolution is a process that acquires adaptations slowly-ish and transmits previously-acquired adaptations to new organisms quickly, while biological evolution is a process that acquires adaptations very slowly and transmits previously-acquired adaptations to new organisms quickly. You seem to be comparing the rate at which cultural evolution acquires adaptations to the rate at which biological evolution transmits previously-acquired adaptations to new organisms, and concluding that cultural evolution is slower.
Re-reading the part of your post where you talked about AI takeoff speeds, you argue (which I hadn’t understood before) that the rise of humans was fast on an evolutionary timescale, and slow on a cultural timescale, so that if it was due to an evolutionary change, it must involve a small change that had a large effect on capabilities, so that a large change will occur very suddenly if we mimic evolution quickly, while if it was due to a cultural change, it was probably a large change, so mimicking culture quickly won’t produce a large effect on capabilities unless it is extremely quick.
This clarifies things, but I don’t agree with the claim. I think slow changes in the intelligence of a species is compatible with fast changes in its capabilities even if the changes are mainly in raw innovative ability rather than cultural learning. Innovations can increase ability to innovate, causing a positive feedback loop. A species could have high enough cultural learning ability for innovations to be transmitted over many generations without having the innovative ability to ever get the innovations that will kick off this loop. Then, when they start slowing gaining innovative ability, the innovations accumulated into cultural knowledge gradually increase, until they reach the feedback loop and the rate of innovation becomes more determined by changes in pre-existing innovations than by changes in raw innovative ability. There doesn’t even have to be any evolutionary changes in the period in which innovation rate starts to get dramatic.
If you don’t buy this story, then it’s not clear why the changes being in cultural learning ability rather than in raw innovative ability would remove the need for a discontinuity. After all, our cultural learning ability went from not giving us much advantage over other animals to “accumulating decisive technological dominance in an evolutionary eyeblink” in an evolutionary eyeblink (quotation marks added for ease of parsing). Does this mean our ability to learn from culture must have greatly increased from a small change? You argue in the post that there’s no clear candidate for what such a discontinuity in cultural learning ability could look like, but this seems just as true to me for raw innovative ability.
Perhaps you could argue that it doesn’t matter if there’s a sharp discontinuity in cultural learning ability because you can’t learn from a culture faster than the culture learns things to teach you. In this case, yes, perhaps I would say that AI-driven culture could make advancements that look discontinuous on a human scale. Though I’m not entirely sure what that would look like, and I admit it does sound kind of soft-takeoffy.