All of these systems are broken, but not because they have a filter, more because the filter works poorly, and moral mazes, etc.
A complementary explanation is that if the system can’t train people (because nobody there knows how to; or because it’s genuinely hard, e.g. despite the Sequences we don’t have a second Yudkowsky), then the only way to find competent people is to filter for outliers. E.g. if you can’t meaningfully raise the IQ of your recruits, instead filter for higher-IQ recruits.
As pointed out in the linked essay, this strategy makes sense if outcomes are heavy-tailed, i.e. if exceptional people provide most of the value. E.g. if an exceptional general is 1000x as valuable as a good one, then it makes sense to filter for exceptional generals; not so much if the difference is only 3x.
For instance by being acknowledged by the first Yudkowsky as the second one. I was referring here mostly to the difficulty of trying to impart expertise from one person to another. Experts can write down and teach legible insights, but the rest of their expertise (which is often the most important stuff) is very hard to teach.
So filter for exceptional loyalty. To which extent that’s worth it depends on how relatively valuable an exceptionally loyal general is to a merely very loyal one, and on the degree to which you can train loyalty.
A complementary explanation is that if the system can’t train people (because nobody there knows how to; or because it’s genuinely hard, e.g. despite the Sequences we don’t have a second Yudkowsky), then the only way to find competent people is to filter for outliers. E.g. if you can’t meaningfully raise the IQ of your recruits, instead filter for higher-IQ recruits.
As pointed out in the linked essay, this strategy makes sense if outcomes are heavy-tailed, i.e. if exceptional people provide most of the value. E.g. if an exceptional general is 1000x as valuable as a good one, then it makes sense to filter for exceptional generals; not so much if the difference is only 3x.
How would you identify a second Yudkowsky? I really don’t like this trope.
By writing ability?
For instance by being acknowledged by the first Yudkowsky as the second one. I was referring here mostly to the difficulty of trying to impart expertise from one person to another. Experts can write down and teach legible insights, but the rest of their expertise (which is often the most important stuff) is very hard to teach.
No military big enough to require multiple layers of general level positions filters for exceptional generals, they all filter for loyalty.
So filter for exceptional loyalty. To which extent that’s worth it depends on how relatively valuable an exceptionally loyal general is to a merely very loyal one, and on the degree to which you can train loyalty.