[Question] Where do (did?) stable, cooperative institutions come from?

The United States has a bunch of nice things whose creation/​maintenance requires coordinated effort from a large number of people across time. For example: bridges that stay up; electrical grids that provide us with power; the rule of law; newspapers that make it easier to keep tabs on recent events; fire fighting services that stop most fires in urban areas; roads; many functioning academic fields; Google; Amazon; grocery stores; the postal service; and so on.

The first question I’d like to pose is: how does this coordination work? What keeps these large sets of people pulling in a common direction (and wanting to pull in a common direction)? And what keeps that “common direction” grounded enough that an actual nice thing results from the pulling (e.g., what causes it to be that you get a working railway system, rather than a bunch of tracks that don’t quite work? what causes you to sometimes get a functioning field of inquiry and not a cargo cult)? Is it that:

  • Many people independently value the nice thing, and they altruistically decide to put their own efforts toward creating/​maintaining the nice thing? (E.g., some large set of people wishes there were good fire-fighting institutions, and so each of them altruistically and independently decides to found a fire-fighting branch, to work at that branch, to tweak that branch’s habits into a more effective configuration, etc.?)

  • A small number of rich and powerful people (who are somehow also knowledgeable about institution design) value the nice thing, and they altruistically decide to set up incentives such that other people, purely via self-interest, will do the work that is needed to create/​maintain the nice thing? (E.g., a small number of people altruistically donate to fire-fighting groups and set up incentives at those groups, and then other people do the fire-fighting work because they want a job?)

  • Something else?

One reason I’d like to pose this question is that it seems plausible to me that the magic that used to enable such cooperative institutions is fading. If so, it seems useful to know about that fading for quite a variety of reasons.

My own lead candidate answer to “what is the magic that lets these cooperative institutions run?” is this:

Somehow, people have sometimes known how to craft “institutional cultures” that aligned an individual’s desire for (glory/​$/​prestige/​etc.) with the actions that will allow the institution as a whole to acquire redistributable (glory, $, prestige, etc.) in the long run. More specifically, cooperative institutions arise in cases where some set of designers (either a few people, or a larger distributed set) magically manage several things at once:

  1. There is an institutional culture that is distinct from the formal workings of the institution, but that exists alongside it, helping to animate it. For example, alongside the formal workings of the old NYT (the printing presses, newspaper subscriptions, staff payroll, explicit assignments, etc.) there was an ethic of journalism that helped direct staff actions at many junctures (an ethic of e.g. “all the news that’s fit to print,” putting in shoe-leather, protecting one’s sources, etc.).

  2. The installed “institutional culture” is pretty good at picking out actions that, if taken, will tend to cause the institution as a whole to gain redistributable (glory/​$/​prestige/​etc.) in the long-term. In our example: The old NYT will in fact gain more long-run prestige, customers, incoming staff talent, etc. if it follows its journalistic ethics. In other words, the culture gestured at by “”all the news that’s fit to print,” putting in shoe-leather, protecting one’s sources, etc.” offered pretty good on-the-ground answers to the question “What can I do now, as an NYT reporter/​manager/​editor/​etc., that will most improve the NYT’s long-term standing?”

  3. The installed “institutional culture” both teaches people how to detect which staff members do/​don’t have that same culture, and prompts people to differentially reward/​punish (and promote/​fire) staff members who do/​don’t have that same culture.

  4. Via 2), an individual staff member will be able to succeed best on personal goals (in terms of some combination of $, prestige, being thought attractive by potential mates, etc.) via following the institutional culture.

I am curious whether this 0-3 account of how stable, cooperative institutions work seems right to you guys (or whether there are caveats, or errors, or important omissions. I’d really like an accurate model here).

Separately (but relatedly – if the above account is importantly wrong, I’ll probably be wrong about this too) – I would like to pose a second question: Is it getting harder to create stable, cooperative institutions in the above sense? If so, why/​how?

Some evidence that it is getting harder:

  • Governmental institutions: There seems to be some degree of institutional failure (mild-ish, so far) in a number of American and especially Californian institutions: California’s electricity is less reliable than it used to be, due basically to bad governance. San Francisco, especially, is seeing rising crime, due more or less to decriminalizing a lot of crime. Many aspects of the covid-19 response also cast our institutions in a worse light than I’d previously anticipated, though it is plausible (given my ignorance) that my anticipations were the silly thing here and that we would not in fact have done better in previous eras. (I’m thinking here of: America being slower than I’d anticipated re: acquiring testing and PPE; putting very little money in the extensive stimulus bill to reducing covid via testing/​research/​etc.; America staying in semi-lockdown for an extended time instead of trying harder either to head toward actual zero (via border control, testing + tracing, etc.) so that we could relax again, or toward something more like herd immunity (while metering it out; but it seems to me that as a country we probably lost more to the costs many parts of America seeming not to lock down for extended periods of time without a plan to use that time to do anything constructive, and without (I think?) adequate accounting for what that would cost in terms of social stability and mental health.)

  • Non-governmental parts of our national sense-making apparatus: Most brand names, e.g. the NYT, Harvard, Science and Nature magazines, the Democrats, the Republicans, the police, the CDC, etc. seem less well-regarded than they used to be. I can’t think of many brands of any sort that are instead better-regarded (Amazon, SpaceX and bitcoin, probably).

  • Subcultures: David Chapman claims that subcultures are much harder to form now /​ more or less don’t exist anymore. I have also tried to look myself, and this matches my own experience: rationality and EA seem among the few things that are sort-of here, and even we are only sort-of here, I think. (“The rationalist diaspora,” not “the happening applied rationality scene.”) (I can think of some others, e.g. the authentic relating /​ circling communities; some other parts of the Thiel-o-sphere; maybe the group at the Stoa; surely some others. But… fewer than I would have expected, and I think fewer than I would have found in past decades?)

  • California/​ the blue coasts have historically been a place where trends originate, then hit the US as a whole, then also the rest of the West. So I’m not sure how local this stuff is or isn’t right now, but I’m worried regardless.

All of this is disputable. And I would love to see your disputes. Even more so, I would love to see your unjustifiable stab-in-the-dark intuitions as to where the center of all this is. From my perspective, the difficulty we are having lately in forming/​sustaining institutional cultures (especially, ones adequate to get much done) seems like one of the central canaries in a puzzle that I badly need to fathom. I’ll put my own hypotheses in the comments.

Acknowledgments: Thanks to lots of folks at Sunday’s town hall discussion for relevant remarks (no fault to them for my errors), and to Ben Hoffman for his essay “Bob the Builder, and the Neo-Puritan Deal.”