Interesting. My main thought reading the OP was simply that coordination is hard at scale, and this applies to intellectual progress too. You had orders-of-magnitude increase in number of people but no change in productivity? Well, did you build better infrastructure and institutions to accommodate, or has your indrastructure for coordinating scientists largely stayed the same since the scientific revolution? In general, scaling early is terrible, and will not be a source of value but run counter to it (and will result in massive goodharting).
So, should we encourage deeper schisms or other differentiation in the rationality community? Ideally not via hurt feelings so we can do occasional cross-pollination.
My cached beliefs from the last time I thought about this is that progress is generally seen in small teams. This is sort of happening naturally as people (in the rationalsphere) tend to cluster into organizations, which are fairly silo’d and/or share their research informally.
This leaves you with the state of “it’s hard for someone not in an org to get up to speed on what that org actually thinks”, and my best guess is to build tools that are genuinely useful for smallish teams and networks to use semi-privately, but which increase the affordance for them to share things with the public (or, larger networks).
I don’t live in the Bay but the way Leverage Research if walled of from the rest of our community seems to be a working way of differentiation that still allows for cross-pollination.
In addition to differentiation within the Bay diffentiation by location would be another good way to have a bit of separation. Funding projects like Kocherga gives us differentation.
I confess, that strategy has not occurred to me. I generally consider improvements to infrastructure to help the group produce value, and if a group has scaled too fast too early, write it off and go elsewhere. Something more (if you’ll forgive the pejorative term) destructive might be right, and perhaps I should consider those strategies more. Though in the case of the rationality community I feel I can see a lot of cheap infrastructure improvements that I think will go a long way to improving coordination of intellectual labour.
(FYI I linked to that comment because I remembered it having a bunch of points about early growth being bad, not because I felt the topic discussed was especially relevant.)
It’s worth considering the effects of the “exploration/exploitation” tradeoff: decreasing coordination/efficiency can increase the efficacy of search in problem space over the long run, precisely because efforts are duplicated. When efforts are duplicated, you increase the probability that someone will find the optimal solution. When everyone is highly coordinated, people all look in the same place and you can end up getting stuck in a “local optimum”—a place that’s pretty good, but can’t be easily improved without scrapping everything and starting over.
It should be noted that I completely buy the “lowest hanging fruit is already picked” explanation. The properties of complex search have been examined somewhat in depth by Stuart Kauffman (“nk space”). These ideas were developed with biological evolution in mind but have been applied to problem solving. In essence, he quantifies the intuition you can improve low-quality things with a lot less search time than it takes to improve high-quality things.
These are precisely the types of spaces in which coordination/efficiency is counterproductive.
I’d be interested in more resources regarding the “low-hanging fruit” theory as related to the structure of ideaspace and how/whether nk space applies to that. Any good resources-for-beginners on Kauffman’s work?
I read “At Home In The Universe: The Search for the Laws of Self Organization and Complexity” which is a very accessible and fun read—I am not a physicist/mathematician/biologist, etc, and it all made sense to me. The book talks about evolution, both biological and technological.
And the model described in that book has been quite commonly adapted by social scientists to study problem solving, so it’s been socially validated as a good framework for thinking about scientific research.
Academia has lots of journals but people submit across journals within a field if they don’t get into their favored one. Likewise if largely overlapping groups are participating in multiple subreddits this seems more like division of labor than the kind of differentiation we saw in the classical Greek city-states.
Interesting. My main thought reading the OP was simply that coordination is hard at scale, and this applies to intellectual progress too. You had orders-of-magnitude increase in number of people but no change in productivity? Well, did you build better infrastructure and institutions to accommodate, or has your indrastructure for coordinating scientists largely stayed the same since the scientific revolution? In general, scaling early is terrible, and will not be a source of value but run counter to it (and will result in massive goodharting).
So, should we encourage deeper schisms or other differentiation in the rationality community? Ideally not via hurt feelings so we can do occasional cross-pollination.
My cached beliefs from the last time I thought about this is that progress is generally seen in small teams. This is sort of happening naturally as people (in the rationalsphere) tend to cluster into organizations, which are fairly silo’d and/or share their research informally.
This leaves you with the state of “it’s hard for someone not in an org to get up to speed on what that org actually thinks”, and my best guess is to build tools that are genuinely useful for smallish teams and networks to use semi-privately, but which increase the affordance for them to share things with the public (or, larger networks).
I don’t live in the Bay but the way Leverage Research if walled of from the rest of our community seems to be a working way of differentiation that still allows for cross-pollination.
In addition to differentiation within the Bay diffentiation by location would be another good way to have a bit of separation. Funding projects like Kocherga gives us differentation.
I confess, that strategy has not occurred to me. I generally consider improvements to infrastructure to help the group produce value, and if a group has scaled too fast too early, write it off and go elsewhere. Something more (if you’ll forgive the pejorative term) destructive might be right, and perhaps I should consider those strategies more. Though in the case of the rationality community I feel I can see a lot of cheap infrastructure improvements that I think will go a long way to improving coordination of intellectual labour.
(FYI I linked to that comment because I remembered it having a bunch of points about early growth being bad, not because I felt the topic discussed was especially relevant.)
The problem with fracturing is that you lose coordination and increase duplication.
I have a more general piece that discusses scaling costs and structure for companies that I think applies here as well—https://www.ribbonfarm.com/2016/03/17/go-corporate-or-go-home/
It’s worth considering the effects of the “exploration/exploitation” tradeoff: decreasing coordination/efficiency can increase the efficacy of search in problem space over the long run, precisely because efforts are duplicated. When efforts are duplicated, you increase the probability that someone will find the optimal solution. When everyone is highly coordinated, people all look in the same place and you can end up getting stuck in a “local optimum”—a place that’s pretty good, but can’t be easily improved without scrapping everything and starting over.
It should be noted that I completely buy the “lowest hanging fruit is already picked” explanation. The properties of complex search have been examined somewhat in depth by Stuart Kauffman (“nk space”). These ideas were developed with biological evolution in mind but have been applied to problem solving. In essence, he quantifies the intuition you can improve low-quality things with a lot less search time than it takes to improve high-quality things.
These are precisely the types of spaces in which coordination/efficiency is counterproductive.
I’d be interested in more resources regarding the “low-hanging fruit” theory as related to the structure of ideaspace and how/whether nk space applies to that. Any good resources-for-beginners on Kauffman’s work?
I read “At Home In The Universe: The Search for the Laws of Self Organization and Complexity” which is a very accessible and fun read—I am not a physicist/mathematician/biologist, etc, and it all made sense to me. The book talks about evolution, both biological and technological.
And the model described in that book has been quite commonly adapted by social scientists to study problem solving, so it’s been socially validated as a good framework for thinking about scientific research.
Subreddits could achieve this without a schism
Academia has lots of journals but people submit across journals within a field if they don’t get into their favored one. Likewise if largely overlapping groups are participating in multiple subreddits this seems more like division of labor than the kind of differentiation we saw in the classical Greek city-states.