Everybody loves a meritocracy until people realize that they’re the ones without merit. I mean you never hear someone say things like:
I think America should be a meritocracy. Ruled by skill rather than personal characteristics or family connections. I mean, I love my son, and he has a great personality. But let’s be real: If we live in a meritocracy he’d be stuck in entry-level.
(I framed the hypothetical this way because I want to exclude senior people very secure in their position who are performatively pushing for meritocracy by saying poor kids are excluded from corporate law or whatever).
In my opinion, if you are serious about meritocracy, you figure out and promote objective tests of competency that a) has high test-retest reliability so you know it’s measuring something real, b) has high predictive validity for the outcome you are interested in getting, and c) has reasonably high accessibility so you know you’re drawing from a wide pool of talent.
For the selection of government officials, the classic Chinese imperial service exam has high (a), low (b), medium (c). For selecting good actors, “Whether your parents are good actors” has maximally high (a), medium-high (b), very low (c). “Whether your startup exited successfully” has low (a), medium-high (b), low (c). The SATs have high (a), medium-low (b), very high (c).
If you’re trying to make society more meritocratic, your number 1 priority should be the design and validation of tests of skill that push the Pareto frontier for various aspects of society, and your number 2 priority should be trying to push for greater incorporation of such tests.
Given that ~ no one really does this, I conclude that very few people are serious about moving towards a meritocracy.
There is a contingent of people who want excellence in education (e.g. Tracing Woodgrains) and are upset about e.g. the deprioritization of math and gifted education and SAT scores in the US. Does that not count?
Given that ~ no one really does this, I conclude that very few people are serious about moving towards a meritocracy.
This sounds like an unreasonably high bar for us humans. You could apply it to all endeavours, and conclude that “very few people are serious about <anything>”. Which is true from a certain perspective, but also stretches the word “serious” far past how it’s commonly understood.
I agree that Tracy does this at a level sufficient to count as “actually care about meritocracy” from my perspective. I would also consider Lee Kuan Yew to actually care a lot about meritocracy, for a more mainstream example.
You could apply it to all endeavours, and conclude that “very few people are serious about <anything>”
Yeah it’s a matter of degree not kind. But I do think many human endeavors pass my bar. I’m not saying people should devote 100% of their efforts to doing the optimal thing. 1-5% done non-optimally seems enough for me, and many people go about that for other activities.
For example, many people care about making (risk-adjusted) returns on their money, and take significant steps towards doing so. For a less facetious example, I think global poverty EAs who earn-to-give or work to make mobile money more accessible count as “actually caring about poverty.”
Similarly, many people say they care about climate change. What do you expect people to do if they care a lot about climate change? Maybe something like
Push for climate-positive policies (including both direct governance and advocacy)
Research or push for better research on climate change
Work on clean energy
Work on getting more nuclear energy
Plant trees and work on other forms of carbon storage
etc (as @Garrett Baker alluded to, someone who thinks a lot about climate change are probably going to have better ideas than me)
We basically see all of these in practice, in significant numbers. Sure, most people who say they care about climate change don’t do any of the above (and (4) is rare, relatively speaking). But the ratio isn’t nearly as dismal as a complete skeptic about human nature would indicate.
Also this conclusion is highly dependent on you, who has thought about this topic for all of 10 minutes, out-thinking the hypothetical people who are actually serious about meritocracy. For example perhaps they do more one-on-one talent scouting or funding, which is indeed very very common and seems to be much more in-demand than psychometric evaluations.
I thought about this for more than 10 minutes, though on a micro rather than macro level (scoped as “how can more competent people work on X” or “how can you hire talented people”). But yeah more like days rather than years.
I think one-on-one talent scouting or funding are good options locally but are much less scalable than psychometric evaluations.
More to the point, I haven’t seen people try to scale those things either. The closest might be something like TripleByte? Or headhunting companies? Certainly when I think of a typical (or 95th-99th percentile) “person who says they care a lot about meritocracy” I’m not imagining a recruiter, or someone in charge of such a firm. Are you?
More to the point, I haven’t seen people try to scale those things either. The closest might be something like TripleByte? Or headhunting companies? Certainly when I think of a typical (or 95th-99th percentile) “person who says they care a lot about meritocracy” I’m not imagining a recruiter, or someone in charge of such a firm. Are you?
I think much of venture capital is trying to scale this thing, and as you said they don’t use the framework you use. The philosophy there is much more oriented towards making sure nobody falls beneath the cracks. Provide the opportunity, then let the market allocate the credit.
That is, the way to scale meritocracy turns out to be maximizing c rather than the other considerations you listed, on current margins.
And then if we say the bottleneck to meritocracy is mostly c rather than a or b, then in fact it seems like our society is absolutely obsessed with making our institutions highly accessible to as broad a pool of talent as possible. There are people who make a whole career out of just advocating for equality.
Mirroring some of the logic in that post, starting from the assumption that neither you nor anyone you know are in the running for a job, (lets say you are hiring an electrician to fix your house) then do you want the person who is going to do a better job or a worse one?
If you are the parent of a child with some kind of developmental problem that means they have terrible hand-eye coordination, you probably don’t want your child to be a brain surgeon, because you can see that is a bad idea.
You do want your child to have resources, and respect and so on. But what they have, and what they do, can be (at least in principle) decoupled. In other words, I think that using a meritocratic system to decide who does what (the people who are good at something should do it) is uncontroversial. However, using a meritocratic system to decide who gets what might be a lot more controversial. For example, as an extreme case you could consider disability benefit for somebody with a mental handicap to be vaguely against the “who gets what” type of meritocracy.
Personally I am strongly in favor of the “who does what” meritocracy, but am kind of neutral on the “who gets what” one.
Makes sense! I agree that this is a valuable place to look. Though I am thinking about tests/assessments in a broader way than you’re framing it here. Eg things that go into this meta-analysis, and improvements/refinements/new ideas, and not just narrow psychometric evaluations.
There’s a also a bit of divergence in “has skills/talent/power” and “cares about what you care about”. Like, yes, maybe there is a very skilled person for that role, but are they trustworthy/reliable/aligned/have the same priorities? You always face the risk of giving some additional power to already powerful adversarial agent. You should be really careful about that. Maybe more focus on the virtue rather than skill.
Shower thought I had a while ago:
Everybody loves a meritocracy until people realize that they’re the ones without merit. I mean you never hear someone say things like:
(I framed the hypothetical this way because I want to exclude senior people very secure in their position who are performatively pushing for meritocracy by saying poor kids are excluded from corporate law or whatever).
In my opinion, if you are serious about meritocracy, you figure out and promote objective tests of competency that a) has high test-retest reliability so you know it’s measuring something real, b) has high predictive validity for the outcome you are interested in getting, and c) has reasonably high accessibility so you know you’re drawing from a wide pool of talent.
For the selection of government officials, the classic Chinese imperial service exam has high (a), low (b), medium (c). For selecting good actors, “Whether your parents are good actors” has maximally high (a), medium-high (b), very low (c). “Whether your startup exited successfully” has low (a), medium-high (b), low (c). The SATs have high (a), medium-low (b), very high (c).
If you’re trying to make society more meritocratic, your number 1 priority should be the design and validation of tests of skill that push the Pareto frontier for various aspects of society, and your number 2 priority should be trying to push for greater incorporation of such tests.
Given that ~ no one really does this, I conclude that very few people are serious about moving towards a meritocracy.
(X-posted)^2
There is a contingent of people who want excellence in education (e.g. Tracing Woodgrains) and are upset about e.g. the deprioritization of math and gifted education and SAT scores in the US. Does that not count?
This sounds like an unreasonably high bar for us humans. You could apply it to all endeavours, and conclude that “very few people are serious about <anything>”. Which is true from a certain perspective, but also stretches the word “serious” far past how it’s commonly understood.
I agree that Tracy does this at a level sufficient to count as “actually care about meritocracy” from my perspective. I would also consider Lee Kuan Yew to actually care a lot about meritocracy, for a more mainstream example.
Yeah it’s a matter of degree not kind. But I do think many human endeavors pass my bar. I’m not saying people should devote 100% of their efforts to doing the optimal thing. 1-5% done non-optimally seems enough for me, and many people go about that for other activities.
For example, many people care about making (risk-adjusted) returns on their money, and take significant steps towards doing so. For a less facetious example, I think global poverty EAs who earn-to-give or work to make mobile money more accessible count as “actually caring about poverty.”
Similarly, many people say they care about climate change. What do you expect people to do if they care a lot about climate change? Maybe something like
Push for climate-positive policies (including both direct governance and advocacy)
Research or push for better research on climate change
Work on clean energy
Work on getting more nuclear energy
Plant trees and work on other forms of carbon storage
etc (as @Garrett Baker alluded to, someone who thinks a lot about climate change are probably going to have better ideas than me)
We basically see all of these in practice, in significant numbers. Sure, most people who say they care about climate change don’t do any of the above (and (4) is rare, relatively speaking). But the ratio isn’t nearly as dismal as a complete skeptic about human nature would indicate.
Also this conclusion is highly dependent on you, who has thought about this topic for all of 10 minutes, out-thinking the hypothetical people who are actually serious about meritocracy. For example perhaps they do more one-on-one talent scouting or funding, which is indeed very very common and seems to be much more in-demand than psychometric evaluations.
I thought about this for more than 10 minutes, though on a micro rather than macro level (scoped as “how can more competent people work on X” or “how can you hire talented people”). But yeah more like days rather than years.
I think one-on-one talent scouting or funding are good options locally but are much less scalable than psychometric evaluations.
More to the point, I haven’t seen people try to scale those things either. The closest might be something like TripleByte? Or headhunting companies? Certainly when I think of a typical (or 95th-99th percentile) “person who says they care a lot about meritocracy” I’m not imagining a recruiter, or someone in charge of such a firm. Are you?
I think much of venture capital is trying to scale this thing, and as you said they don’t use the framework you use. The philosophy there is much more oriented towards making sure nobody falls beneath the cracks. Provide the opportunity, then let the market allocate the credit.
That is, the way to scale meritocracy turns out to be maximizing c rather than the other considerations you listed, on current margins.
And then if we say the bottleneck to meritocracy is mostly c rather than a or b, then in fact it seems like our society is absolutely obsessed with making our institutions highly accessible to as broad a pool of talent as possible. There are people who make a whole career out of just advocating for equality.
There was an interesting Astral Codex 10 thing related to this kind of idea: https://www.astralcodexten.com/p/book-review-the-cult-of-smart
Mirroring some of the logic in that post, starting from the assumption that neither you nor anyone you know are in the running for a job, (lets say you are hiring an electrician to fix your house) then do you want the person who is going to do a better job or a worse one?
If you are the parent of a child with some kind of developmental problem that means they have terrible hand-eye coordination, you probably don’t want your child to be a brain surgeon, because you can see that is a bad idea.
You do want your child to have resources, and respect and so on. But what they have, and what they do, can be (at least in principle) decoupled. In other words, I think that using a meritocratic system to decide who does what (the people who are good at something should do it) is uncontroversial. However, using a meritocratic system to decide who gets what might be a lot more controversial. For example, as an extreme case you could consider disability benefit for somebody with a mental handicap to be vaguely against the “who gets what” type of meritocracy.
Personally I am strongly in favor of the “who does what” meritocracy, but am kind of neutral on the “who gets what” one.
The field you should look at I think is Industrial and Organizational Psychology, as well as the classic Item Response Theory.
Makes sense! I agree that this is a valuable place to look. Though I am thinking about tests/assessments in a broader way than you’re framing it here. Eg things that go into this meta-analysis, and improvements/refinements/new ideas, and not just narrow psychometric evaluations.
There’s a also a bit of divergence in “has skills/talent/power” and “cares about what you care about”. Like, yes, maybe there is a very skilled person for that role, but are they trustworthy/reliable/aligned/have the same priorities? You always face the risk of giving some additional power to already powerful adversarial agent. You should be really careful about that. Maybe more focus on the virtue rather than skill.