I think the last parahraph of your comment agrees with the post:
″… if you try to measure skill and then pick people based on how well they performed on your measure, you’re actually selecting for skill + luck rather than pure skill.”
Right, but that phrasing is misleading. Selecting by pure skill isn’t available, and a lot of luck might be a hidden skill. Selecting by outcome is NOT luck-free, but it’s better than any other selection mechanism.
The problem isn’t the selecting by outcome part, the problem is the part where one tries to discriminate more than the measure allows.
Suppose that you are making people take an exam on which they can score between 0 and 100. If you took their exam scores literally as indicative of their skill, you might be tempted to use the scores to choose the people in the top 1% (say). But the exam probably isn’t capable of measuring skill that precisely; the error of the measure may be such that realistically, it’s only reliably picking up on skill differences on the level of 10-point increments. If you try to use it to pick the top 1% anyway, you are just measuring noise.
So if you want to make things fair in the sense of rewarding everyone who actually has enough skill, you should set the cutoff so that everyone who scores in the top decentile gets rewarded. (This does mean that some people who actually aren’t that skilled will also be rewarded by luck, but that’s an unavoidable trade-off.)
If people didn’t respond to incentives, then you wouldn’t need to care about making things fair in this sense. After all, anyone who’s in the top 1% is in the top 10% too, and picking the people in the top 1% does mean that you’re very unlikely to get someone who’s actually in the top 89% but made it over the 90% cutoff by luck. But people do respond to incentives; in particular, if they realize that they can’t make it to your cutoff just by being talented and hard-working but have to be exceptionally lucky as well, then they will feel the pressure to cheat and deliver results that look good but are actually false.
In fact, if this is common enough, then picking the people that the measure says are in the top 1% may actually get you people who are worse, at least in the sense of being more willing to fake their results, if it’s more likely that you’ll get into the top 1% by cheating than by being lucky.
I find it useful to think of this literally as an exam situation: you know that you need to score in the top percentile to be rewarded. You also know that skill only gets anyone over the 90 point mark and the rest is luck. Furthermore, you know that a number of people also taking the exam—some equally skilled as you, some less—are going to cheat on it, and that the vast majority of people who do make it to the top percentile only succeed at that because they cheated. If you don’t cheat, it doesn’t matter how skilled or hard-working you are: you almost certainly won’t make it.
Would you cheat?
If no, you’ll get eliminated from the system and have to do something else. If yes, you’re contributing to the situation and increasing the incentive for others to cheat, and helping establish implicit norms where it’s taken for granted that anyone who made it past this round of elimination is a cheater, so cheating can’t be that bad.
Whereas if the cutoff was at the 90th percentile level, some people would still get in because they were cheats, but the system would no longer be selecting strongly against people with integrity, and the environment would be much more conducive for establishing norms where cheating was looked unfavorably on.
I think there’s a crux here we haven’t made explicit. I don’t think the universe is particularly fair, and I don’t intend to give up much efficiency in order to correct that unfairness. I don’t want to reward skill, I want to eat fresher strawberries in the winter. I often signal to people that the best way to get my resources is to have and develop skill, but this doesn’t contradict the fact that there’s a lot of luck which neither I nor they can control.
I don’t think most economic choices are very similar to school tests. They are simply not of the form “try to find the hidden intrinsic values of someone”. No single human decides the grading mechanism, and it’s not arbitrary proof of work to some authority. Economic choices are more of an uncertain time/energy/resource allocation problem—we MUST make choices of how to (try to) provide value to someone with resources, and how to expend those resources to best get the things we want. For this, even noisy measures are better than random. It’s the best (and most resistant to Goodheart’s Law) predictor of future delivery of value we have—even if it’s mostly noise, there’s some signal there.
My fear with UBI (note that on balance I’m still in favor) is that it doesn’t change any fundamentals here—all these choices are still noisy and luck-entangled—it just removes a little bit of signal, making it a little bit less likely to optimize the outcomes.
For this, even noisy measures are better than random. It’s the best (and most resistant to Goodheart’s Law) predictor of future delivery of value we have—even if it’s mostly noise, there’s some signal there.
I agree, and don’t think that I’ve said anything that would imply otherwise.
it just removes a little bit of signal, making it a little bit less likely to optimize the outcomes.
Hmm, how does it remove a little bit of signal? To me, the signal of an academic’s worth is their publication/research history; the existence of a UBI is not going to remove that.
I still resist the idea that this is a signal of worth, and certainly that it’s a signal of worth in a field. It’s a signal of demand for a given output and of what an individual should do in order to give value to other humans.
The signal is _not_ how good of an academic one is. The signal is whether more academics are needed, vs other professions entirely. This signal is somewhat muted if the recipient is less dependent on market income.
Ah, we have been talking about different signals then.
I think we should be careful about lost purposes here: the function of an economic system should, IMO, be to ensure that everyone has the possibility to live a life that they experience as flourishing, while distracting them from that goal as little possible. If the productivity of an economic system is such that there’s a sizable group of people who are happy and capable of having a flourishing life with a minimum wage-equivalent UBI, then that’s a sign that the economic system is starting to be done. Not that it would be done—obviously it wouldn’t work yet if everyone just lived off the UBI and only did things that they personally found meaningful. But those people are starting to pave the way towards a broader societal change that allows a life only doing personally meaningful things, as their presence and example will encourage others to make the same transition and also create cultural institutions that support such a lifestyle, paving the way for a broader transition to such a society. Including structures that support people doing socially-needed things due to intrinsic motivation and because voluntarily doing useful things becomes valued, rather than socially-needed things only being done because you get money for them.
My model here would take too long to explain in detail, but briefly, I suspect that a lot of the thing about jobs being something that you only do grudgingly is because people have been forced from an early age to do things they don’t want to and come to hate it, whereas much more work would be done out of a volunteer basis and because people want to be useful if people had grown up in a society where the opportunity to do the-kinds-of-things-we-now-think-of-as-work was considered a joy and a privilege. See e.g. the research on SDT and how relatedness and competence are basic needs behind intrinsic motivation, but how extrinsic motivation tends to suppress intrinsic motivation.
an economic system should, IMO, be to ensure that everyone has the possibility to live a life that they experience as flourishing, while distracting them from that goal as little possible
Oh, yeah, that’s definitely a point of confusion between us. IMO, an economic system is an evolved mechanism for communicating individual desires and negotiating behavior and resource constraints without having to pay the cost of the underlying violence behind the idea of ownership and the enforcement of necessary cooperation.
I broadly agree that we’re in an unpleasant equilibrium of what’s “necessary” and “unpleasant”. I don’t have a picture of a better equilibrium that supports this density of humans with the same distribution of capability and the same or better amount of individual autonomy. I prefer that there are enough porta-potty cleaners who choose that as their self-actualized goal, but I don’t have a lot of faith that it’s possible with current levels of human development.
I think the last parahraph of your comment agrees with the post:
″… if you try to measure skill and then pick people based on how well they performed on your measure, you’re actually selecting for skill + luck rather than pure skill.”
Right, but that phrasing is misleading. Selecting by pure skill isn’t available, and a lot of luck might be a hidden skill. Selecting by outcome is NOT luck-free, but it’s better than any other selection mechanism.
The problem isn’t the selecting by outcome part, the problem is the part where one tries to discriminate more than the measure allows.
Suppose that you are making people take an exam on which they can score between 0 and 100. If you took their exam scores literally as indicative of their skill, you might be tempted to use the scores to choose the people in the top 1% (say). But the exam probably isn’t capable of measuring skill that precisely; the error of the measure may be such that realistically, it’s only reliably picking up on skill differences on the level of 10-point increments. If you try to use it to pick the top 1% anyway, you are just measuring noise.
So if you want to make things fair in the sense of rewarding everyone who actually has enough skill, you should set the cutoff so that everyone who scores in the top decentile gets rewarded. (This does mean that some people who actually aren’t that skilled will also be rewarded by luck, but that’s an unavoidable trade-off.)
If people didn’t respond to incentives, then you wouldn’t need to care about making things fair in this sense. After all, anyone who’s in the top 1% is in the top 10% too, and picking the people in the top 1% does mean that you’re very unlikely to get someone who’s actually in the top 89% but made it over the 90% cutoff by luck. But people do respond to incentives; in particular, if they realize that they can’t make it to your cutoff just by being talented and hard-working but have to be exceptionally lucky as well, then they will feel the pressure to cheat and deliver results that look good but are actually false.
In fact, if this is common enough, then picking the people that the measure says are in the top 1% may actually get you people who are worse, at least in the sense of being more willing to fake their results, if it’s more likely that you’ll get into the top 1% by cheating than by being lucky.
I find it useful to think of this literally as an exam situation: you know that you need to score in the top percentile to be rewarded. You also know that skill only gets anyone over the 90 point mark and the rest is luck. Furthermore, you know that a number of people also taking the exam—some equally skilled as you, some less—are going to cheat on it, and that the vast majority of people who do make it to the top percentile only succeed at that because they cheated. If you don’t cheat, it doesn’t matter how skilled or hard-working you are: you almost certainly won’t make it.
Would you cheat?
If no, you’ll get eliminated from the system and have to do something else. If yes, you’re contributing to the situation and increasing the incentive for others to cheat, and helping establish implicit norms where it’s taken for granted that anyone who made it past this round of elimination is a cheater, so cheating can’t be that bad.
Whereas if the cutoff was at the 90th percentile level, some people would still get in because they were cheats, but the system would no longer be selecting strongly against people with integrity, and the environment would be much more conducive for establishing norms where cheating was looked unfavorably on.
I think there’s a crux here we haven’t made explicit. I don’t think the universe is particularly fair, and I don’t intend to give up much efficiency in order to correct that unfairness. I don’t want to reward skill, I want to eat fresher strawberries in the winter. I often signal to people that the best way to get my resources is to have and develop skill, but this doesn’t contradict the fact that there’s a lot of luck which neither I nor they can control.
I don’t think most economic choices are very similar to school tests. They are simply not of the form “try to find the hidden intrinsic values of someone”. No single human decides the grading mechanism, and it’s not arbitrary proof of work to some authority. Economic choices are more of an uncertain time/energy/resource allocation problem—we MUST make choices of how to (try to) provide value to someone with resources, and how to expend those resources to best get the things we want. For this, even noisy measures are better than random. It’s the best (and most resistant to Goodheart’s Law) predictor of future delivery of value we have—even if it’s mostly noise, there’s some signal there.
My fear with UBI (note that on balance I’m still in favor) is that it doesn’t change any fundamentals here—all these choices are still noisy and luck-entangled—it just removes a little bit of signal, making it a little bit less likely to optimize the outcomes.
I agree, and don’t think that I’ve said anything that would imply otherwise.
Hmm, how does it remove a little bit of signal? To me, the signal of an academic’s worth is their publication/research history; the existence of a UBI is not going to remove that.
I still resist the idea that this is a signal of worth, and certainly that it’s a signal of worth in a field. It’s a signal of demand for a given output and of what an individual should do in order to give value to other humans.
The signal is _not_ how good of an academic one is. The signal is whether more academics are needed, vs other professions entirely. This signal is somewhat muted if the recipient is less dependent on market income.
Ah, we have been talking about different signals then.
I think we should be careful about lost purposes here: the function of an economic system should, IMO, be to ensure that everyone has the possibility to live a life that they experience as flourishing, while distracting them from that goal as little possible. If the productivity of an economic system is such that there’s a sizable group of people who are happy and capable of having a flourishing life with a minimum wage-equivalent UBI, then that’s a sign that the economic system is starting to be done. Not that it would be done—obviously it wouldn’t work yet if everyone just lived off the UBI and only did things that they personally found meaningful. But those people are starting to pave the way towards a broader societal change that allows a life only doing personally meaningful things, as their presence and example will encourage others to make the same transition and also create cultural institutions that support such a lifestyle, paving the way for a broader transition to such a society. Including structures that support people doing socially-needed things due to intrinsic motivation and because voluntarily doing useful things becomes valued, rather than socially-needed things only being done because you get money for them.
My model here would take too long to explain in detail, but briefly, I suspect that a lot of the thing about jobs being something that you only do grudgingly is because people have been forced from an early age to do things they don’t want to and come to hate it, whereas much more work would be done out of a volunteer basis and because people want to be useful if people had grown up in a society where the opportunity to do the-kinds-of-things-we-now-think-of-as-work was considered a joy and a privilege. See e.g. the research on SDT and how relatedness and competence are basic needs behind intrinsic motivation, but how extrinsic motivation tends to suppress intrinsic motivation.
Oh, yeah, that’s definitely a point of confusion between us. IMO, an economic system is an evolved mechanism for communicating individual desires and negotiating behavior and resource constraints without having to pay the cost of the underlying violence behind the idea of ownership and the enforcement of necessary cooperation.
I broadly agree that we’re in an unpleasant equilibrium of what’s “necessary” and “unpleasant”. I don’t have a picture of a better equilibrium that supports this density of humans with the same distribution of capability and the same or better amount of individual autonomy. I prefer that there are enough porta-potty cleaners who choose that as their self-actualized goal, but I don’t have a lot of faith that it’s possible with current levels of human development.