For my own benefit I thought I’d write down examples of markets that I can see are inadequate yet inexploitable. Not all of these I’m sure are actually true, some just fit the pattern.
I notice that most charities aren’t cost effective, but if I decide to do better by making a super cost-effective charity I shouldn’t expect to be more successful than the other charities.
I notice that most people at university aren’t trying to learn but get good signals for their career, I can’t easily do better in the job market by stopping trying to signal and just learn better
I notice most parenting technique books aren’t helpful (because genetics), but I probably can’t make money by selling a shorter book that tells you the only parenting techniques that do matter.
If I notice that politicians aren’t trying to improve the country very much, I can’t get elected over them by just optimising for improving the country more (because they’re optimising for being elected).
If most classical musicians spend a lot of money on high-status instruments and spend time with high-status teachers that don’t correlate with quality, you can’t be more successful by just picking high quality instruments and teachers.
If most rocket companies are optimising for getting the most money out of government, you probably can’t win government contracts by just making a better rocket company. (?)
If I notice that nobody seems to be doing research on the survival of the human species, I probably can’t make it as an academic by making that my focus
If I notice that most music recommendation sites are highly reviewing popular music (so that they get advance copies) I can’t have a more successful review site/magazine by just being honest about the music.
Correspondingly, if these models are true, here are groups/individuals who it would be a mistake to infer strong information about if they’re not doing well in these markets:
Just because a charity has a funding gap doesn’t mean it’s not very cost-effective
Just because someone has bad grades at university doesn’t mean they are bad at learning their field
Just because a parenting book isn’t selling well doesn’t mean it isn’t more useful than others
Just because a politician didn’t get elected doesn’t mean they wouldn’t have made better decisions
Just because a rocket company doesn’t get a government contract doesn’t mean it isn’t better at building safe and cheap rockets than other companies
Just because an academic is low status / outside academia doesn’t mean they’re views aren’t true
Just because a band isn’t highly reviewed in major publications doesn’t mean it isn’t innovative/great
Some of these seem stronger to me than others. I tend to think that academic fields are more adequate at finding truth and useful knowledge than music critics are adequate at figuring out which bands are good.
I notice that most charities aren’t cost effective, but if I decide to do better by making a super cost-effective charity I shouldn’t expect to be more successful than the other charities.
This seems wrong to me. I think you should expect to be more cost-effective, but you should also expect to get much less funding than the average charity (all else equal), which might still make the total impact you have larger.
Or to phrase it in Eliezer’s jargon: The market of charities is exploitable in respect to cost-effectiveness, but inexploitable in respect to funding. And ultimately you care about cost-effectiveness * funding.
I tend to think that academic fields are more adequate at finding truth and useful knowledge than music critics are adequate at figuring out which bands are good.
Why is that?
Presumably there’s a lot of money on the line to identify music that people will like. (Though I guess that’s the role of studio execs and producers, and not critics.)
It turns out it is possible to compete by playing a different game than the incumbents, but you need to have a huge amount of capital and tolerate the risk of losing all of it. If you read the early history of SpaceX, they came within a hair’s width of bankruptcy.
This seems to be a general feature of startup founder stories. Founding a startup has been so romanticized that some people would still try even if there’s only a 1% chance of success. So even though it requires individuals to go against the energy-gradient, we nevertheless see people try to innovate. And a few even succeed. And this is good for society.
This might give us a general tactic for breaking out of inadequacy-traps: if the successful risk-takers are awarded enough status, then others will try to follow their example even though the expected utility of doing so is disasterous for the individual.
I’ve heard it said that the fastest way to win a Nobel prize is to do something that everyone else in your field believed was impossible. Maybe academia recognizes that it’s held back by a bandwagon-research inadequacy-trap, and giving its highest prize to researchers that don’t follow the same incentives as everyone else is a way that the system self-corrects? Not by compensating them for their risk, but by encouraging other researchers to act against their own best interests and perform risky research in the vain hope of obtaining a ludicrously high status prize.
I don’t know how you use this to fix politics, though. My first thought is to declare some genuine lifelong public servants to be national heroes or something, but I worry that any such honor will become a popularity contest that the more common kind of politician is more optimized for.
I’d argue the complexity of information gathering and crappy UI of voter punishment or reward are more relevant to politics.
A good model of where to start might be an efficient market of many educated actors being able to fix the political power of polticians the same way current markets fix the price of stocks today.
There’s already a relatively open field for actors willing to become journalists or podcasters so the media moving piece in the current system is less systematically broken. It’s also a component in sufficient other systems that are less broken than politics that we should expect it possible to keep the current media and still have better efficiency that today be attaignable.
Not sure how to implement the specifics, however...
For my own benefit I thought I’d write down examples of markets that I can see are inadequate yet inexploitable. Not all of these I’m sure are actually true, some just fit the pattern.
I notice that most charities aren’t cost effective, but if I decide to do better by making a super cost-effective charity I shouldn’t expect to be more successful than the other charities.
I notice that most people at university aren’t trying to learn but get good signals for their career, I can’t easily do better in the job market by stopping trying to signal and just learn better
I notice most parenting technique books aren’t helpful (because genetics), but I probably can’t make money by selling a shorter book that tells you the only parenting techniques that do matter.
If I notice that politicians aren’t trying to improve the country very much, I can’t get elected over them by just optimising for improving the country more (because they’re optimising for being elected).
If most classical musicians spend a lot of money on high-status instruments and spend time with high-status teachers that don’t correlate with quality, you can’t be more successful by just picking high quality instruments and teachers.
If most rocket companies are optimising for getting the most money out of government, you probably can’t win government contracts by just making a better rocket company. (?)
If I notice that nobody seems to be doing research on the survival of the human species, I probably can’t make it as an academic by making that my focus
If I notice that most music recommendation sites are highly reviewing popular music (so that they get advance copies) I can’t have a more successful review site/magazine by just being honest about the music.
Correspondingly, if these models are true, here are groups/individuals who it would be a mistake to infer strong information about if they’re not doing well in these markets:
Just because a charity has a funding gap doesn’t mean it’s not very cost-effective
Just because someone has bad grades at university doesn’t mean they are bad at learning their field
Just because a parenting book isn’t selling well doesn’t mean it isn’t more useful than others
Just because a politician didn’t get elected doesn’t mean they wouldn’t have made better decisions
Just because a rocket company doesn’t get a government contract doesn’t mean it isn’t better at building safe and cheap rockets than other companies
Just because an academic is low status / outside academia doesn’t mean they’re views aren’t true
Just because a band isn’t highly reviewed in major publications doesn’t mean it isn’t innovative/great
Some of these seem stronger to me than others. I tend to think that academic fields are more adequate at finding truth and useful knowledge than music critics are adequate at figuring out which bands are good.
This seems wrong to me. I think you should expect to be more cost-effective, but you should also expect to get much less funding than the average charity (all else equal), which might still make the total impact you have larger.
Or to phrase it in Eliezer’s jargon: The market of charities is exploitable in respect to cost-effectiveness, but inexploitable in respect to funding. And ultimately you care about cost-effectiveness * funding.
Yup—I was under-specific about the definition of ‘successful’. Thanks!
Why is that?
Presumably there’s a lot of money on the line to identify music that people will like. (Though I guess that’s the role of studio execs and producers, and not critics.)
To echo Meister’s comment, I think that rocket companies are different because of the high activation energy. It is not easy money.
Re: Rocket Companies
It turns out it is possible to compete by playing a different game than the incumbents, but you need to have a huge amount of capital and tolerate the risk of losing all of it. If you read the early history of SpaceX, they came within a hair’s width of bankruptcy.
This seems to be a general feature of startup founder stories. Founding a startup has been so romanticized that some people would still try even if there’s only a 1% chance of success. So even though it requires individuals to go against the energy-gradient, we nevertheless see people try to innovate. And a few even succeed. And this is good for society.
This might give us a general tactic for breaking out of inadequacy-traps: if the successful risk-takers are awarded enough status, then others will try to follow their example even though the expected utility of doing so is disasterous for the individual.
I’ve heard it said that the fastest way to win a Nobel prize is to do something that everyone else in your field believed was impossible. Maybe academia recognizes that it’s held back by a bandwagon-research inadequacy-trap, and giving its highest prize to researchers that don’t follow the same incentives as everyone else is a way that the system self-corrects? Not by compensating them for their risk, but by encouraging other researchers to act against their own best interests and perform risky research in the vain hope of obtaining a ludicrously high status prize.
I don’t know how you use this to fix politics, though. My first thought is to declare some genuine lifelong public servants to be national heroes or something, but I worry that any such honor will become a popularity contest that the more common kind of politician is more optimized for.
I’d argue the complexity of information gathering and crappy UI of voter punishment or reward are more relevant to politics. A good model of where to start might be an efficient market of many educated actors being able to fix the political power of polticians the same way current markets fix the price of stocks today. There’s already a relatively open field for actors willing to become journalists or podcasters so the media moving piece in the current system is less systematically broken. It’s also a component in sufficient other systems that are less broken than politics that we should expect it possible to keep the current media and still have better efficiency that today be attaignable.
Not sure how to implement the specifics, however...