Responding to Chris: if you go look at real bureaucracies, it is not really the case that “at each level the bosses tell the subordinates what to do and they just have to do it”. At every bureaucracy I’ve worked in/around, lower-level decision makers had many de facto degrees of freedom. You can think of this as a generalization of one of the central problems of jurisprudence: in practice, human “bosses” (or legislatures, in the jurisprudence case) are not able to give instructions which unambiguously specify what to do in all the crazy situations which come up in practice. Nor do people at the top have anywhere near the bandwidth needed to decide every ambiguous case themselves; there is far too much ambiguity in the world. So, in practice, lower-level people (i.e. judges at various levels) necessarily make many many judgement calls in the course of their work.
Also, in general, tons of information flows back up the hierarchy for higher-level people to make decisions. There are already bureacracies whose purpose is very similar to HCH: they exist to support the decision-making of the person at the top. (Government intelligence is a good example.) To my knowledge/experience, such HCH-like bureacracies are not any less dysfunctional than others, nor do normal bureacracies behave less dysfunctionally than normal when passing information up to a high-level decision maker.
Responding to Joe: if you go look at real bureaucracies, most people working in them are generally well-meaning and trying to help. There is still a sense in which incentives are a limiting factor: good incentives are information-carriers in their own right (like e.g. prices), and I’ll link below to arguments that information-transmission is the problem. But incentives are not the problem in a way which can be fixed just by having everyone share some non-selfish values.
So why do bureaucracies (and large organizations more generally) fail so badly?
My main model for this is that interfaces are a scarce resource. Or, to phrase it in a way more obviously relevant to factorization: it is empirically hard for humans to find good factorizations of problems which have not already been found. Interfaces which neatly split problems are not an abundant resource (at least relative to humans’ abilities to find/build such interfaces). If you can solve that problem well, robustly and at scale, then there’s an awful lot of money to be made.
Also, one major sub-bottleneck (though not the only sub-bottleneck) of interface scarcity is that it’s hard to tell who has done a good job on a domain-specific problem/question without already having some domain-specific background knowledge. This also applies at a more “micro” level: it’s hard to tell whose answers are best without knowing lots of context oneself.
I should also mention: these models came out of me working in/around bureacratic organizations, as they were trying to scale up. I wanted to generally understand the causes of various specific instances of dysfunction. So they are based largely on first-hand knowledge.
John Wentworth on bureaucracy: