> If you’re a king with 5 districts, and you have 20 competent managers who trust each other… one thing you can do is assign 4 competent managers to each fortress, to ensure the fortress has redundancy and resilience and to handle all of its business without any backstabbing or relying on inflexible bureaucracies. But another thing you can do is send 10 (or 15!) of the managers to conquer and reign over *another* 5 (or 15!) districts.
> This is bad if you’re one of the millions of people who live in the kingdom, who have to contend with werewolves.
> It’s an acceptable price to pay if you’re actually the king. Because if you didn’t pay the price, you’d be outcompeted by an empire who did. And meanwhile it doesn’t actually really affect your plans that much.
...
The key instinct is that any price that can be paid to be stronger or more competitive, must be paid, therefore despair: If you didn’t pay the price, you’d be out-competed by someone who did. People who despair this way often intuitively are modeling things as effectively perfect competition at least over time, which causes them to think that everything must by default become terrible, likely right away.
[...]
Kingdoms don’t reliably expand to their breaking points.
Anthropics vs Goals
I didn’t get around to replying to this until today, but this wasn’t my main point and I think it’s pretty important.
The issue isn’t whether you’ll fail to achieve your goals if you don’t expand. The issue is “from an anthropic reasoning perspective, what sort of world will most people live in?”
I have shifted some of my thinking around “you’ll be outcompeted and therefore it’s in your interest to expand”. I think I agree with “it’s generally not worth trying to be the winner-take-all winner, because a) you need to sacrifice all the things you cared about anyway, b) even if you do, you’re not actually likely to win anyway.”
But that was only half the question – if you’re looking around the world, trying to build a model of what’s going on, I think the causal explanation is that “organizations that expand end up making up most of the world, so they’ll account for most of your observations.”
The reason this seems important is, like, I see you and Benquo looking in horror at the world. And… it is a useful takeaway that “hmm, I guess I don’t need to expand in order to compete with the biggest empires in order to be happy/successful/productive, I can just focus on making a good business that delivers value and doesn’t compromise it’s integrity.” (And having that crystallized has been helpful to my own developing worldview)
Nonetheless… the world will continue to be a place you recoil in horror from until somehow, someone creates something either stops mazes, or outcompetes them, or something.
Breaking Points vs Realistic Tradeoffs
I also disagree with the characterization “kingdoms don’t expect to breaking point.”
The original concept here was “why do people have a hard time detecting obfuscators and sociopaths?”. A realistic example (to be clear I don’t know much about medieval kingdoms), is a corporation that ends up creating multiple departments (i.e. hiring a legal team), or expanding to new locations.
This doesn’t mean you expand to your breaking point – any longterm organization has to contend with shocks and robustness. The organizations I expect to be most successful will expand carefully, not overextending. But if you’re asking the question “why are there obfuscators everywhere?”, I think the answer is because the relative profitability of extinguishing obfusctators, vs. not worrying as much about it, points towards the latter direction.
This is, in part, because extinguishing obfuscating or other mazelike patterns is a rare, high skill job that, like, even small organizations don’t usually have the capacity to deal with. I think if you can make it much cheaper, it’s probably possible to shift the global pattern. But I think the status quo is that the profit-maximizing thing to do focus on expansion over being maze-proof, and there’s a lot of profit-maximizing-entities out there.
It’s not worth it for the king to try to expand to take over the world. It still seems, for many kings in many places, that expanding reasonably, robustly, is still the right strategy given their goals (or at least, they think it’s their goal, and you’d have your work cut out for you convincing them otherwise), and that meanwhile worrying about werewolves in the lawyer department is probably more like a form of altruism than a form of self-interest.
Or, reframing this as a question (since I’m honestly not that confident)
If your inner circle is safe, how much selfish reason does a CEO have to make sure the rest of the organization is obfuscator-proof?
Anthropics vs Goals
I didn’t get around to replying to this until today, but this wasn’t my main point and I think it’s pretty important.
The issue isn’t whether you’ll fail to achieve your goals if you don’t expand. The issue is “from an anthropic reasoning perspective, what sort of world will most people live in?”
I have shifted some of my thinking around “you’ll be outcompeted and therefore it’s in your interest to expand”. I think I agree with “it’s generally not worth trying to be the winner-take-all winner, because a) you need to sacrifice all the things you cared about anyway, b) even if you do, you’re not actually likely to win anyway.”
But that was only half the question – if you’re looking around the world, trying to build a model of what’s going on, I think the causal explanation is that “organizations that expand end up making up most of the world, so they’ll account for most of your observations.”
The reason this seems important is, like, I see you and Benquo looking in horror at the world. And… it is a useful takeaway that “hmm, I guess I don’t need to expand in order to compete with the biggest empires in order to be happy/successful/productive, I can just focus on making a good business that delivers value and doesn’t compromise it’s integrity.” (And having that crystallized has been helpful to my own developing worldview)
Nonetheless… the world will continue to be a place you recoil in horror from until somehow, someone creates something either stops mazes, or outcompetes them, or something.
Breaking Points vs Realistic Tradeoffs
I also disagree with the characterization “kingdoms don’t expect to breaking point.”
The original concept here was “why do people have a hard time detecting obfuscators and sociopaths?”. A realistic example (to be clear I don’t know much about medieval kingdoms), is a corporation that ends up creating multiple departments (i.e. hiring a legal team), or expanding to new locations.
This doesn’t mean you expand to your breaking point – any longterm organization has to contend with shocks and robustness. The organizations I expect to be most successful will expand carefully, not overextending. But if you’re asking the question “why are there obfuscators everywhere?”, I think the answer is because the relative profitability of extinguishing obfusctators, vs. not worrying as much about it, points towards the latter direction.
This is, in part, because extinguishing obfuscating or other mazelike patterns is a rare, high skill job that, like, even small organizations don’t usually have the capacity to deal with. I think if you can make it much cheaper, it’s probably possible to shift the global pattern. But I think the status quo is that the profit-maximizing thing to do focus on expansion over being maze-proof, and there’s a lot of profit-maximizing-entities out there.
It’s not worth it for the king to try to expand to take over the world. It still seems, for many kings in many places, that expanding reasonably, robustly, is still the right strategy given their goals (or at least, they think it’s their goal, and you’d have your work cut out for you convincing them otherwise), and that meanwhile worrying about werewolves in the lawyer department is probably more like a form of altruism than a form of self-interest.
Or, reframing this as a question (since I’m honestly not that confident)
If your inner circle is safe, how much selfish reason does a CEO have to make sure the rest of the organization is obfuscator-proof?