You must misunderstand me. To what you say, I say that you don’t want your org to be fighting the incentives of the environment around it. You want to set up your org in a position in the environment where the incentives within the org correlate with doing good. If the founders of Nvidia didn’t want marginally better GPUs to be made, then they hired the wrong people, bought the wrong infrastructure, partnered with the wrong companies, and overall made the wrong organizational incentive structure for that job.
I would in fact be surprised if there were >1k worker sized orgs which consistently didn’t reward their workers for doing good according to the org’s values, was serving no demand present in the market, and yet were competently executing some altruistic goal.
Right now I feel like I’m just saying a bunch of obvious things which you should definitely agree with, yet you believe we have a disagreement. I do not understand what you think I’m saying. Maybe you could try restating what I originally said in your own words?
We absolutely agree that incentives matter. Where I think we disagree is on how much they matter and how controllable they are. Especially for orgs whose goals are orthogonal or even contradictory with the common cultural and environmental incentives outside of the org.
I’m mostly reacting to your topic sentence
EAs are, and I thought this even before the recent Altman situation, strikingly bad at setting up good organizational incentives.
And wondering if ‘strikingly bad’ is relative to some EA or non-profit-driven org that does it well,or if ‘strikingly bad’ is just acknowledgement that it may not be possible to do well.
By strikingly bad I mean there are easy changes EA can make to make it’s sponsored orgs have better incentives, and it has too much confidence that the incentives in the orgs it sponsors favor doing good above doing bad, politics, not doing anything, etc.
For example, nobody in Anthropic gets paid more if they follow their RSP and less of they don’t. Changing this isn’t sufficient for me to feel happy with Anthropic, but its one example among many for which Anthropic could be better.
When I think of an Anthropic I feel happy with I think of a formally defined balance of powers type situation with strong & public whistleblower protection and post-whistleblower reform processes, them hiring engineers loyal to that process (rather than building AGI), and them diversifying the sources for which they trade, such that its in none of their source’s interest to manipulate them.
I also claim marginal movements toward this target are often good.
As I said in the original shortform, I also think incentives are not all or nothing. Worse incentives just mean you need more upstanding workers & leaders.
You must misunderstand me. To what you say, I say that you don’t want your org to be fighting the incentives of the environment around it. You want to set up your org in a position in the environment where the incentives within the org correlate with doing good. If the founders of Nvidia didn’t want marginally better GPUs to be made, then they hired the wrong people, bought the wrong infrastructure, partnered with the wrong companies, and overall made the wrong organizational incentive structure for that job.
I would in fact be surprised if there were >1k worker sized orgs which consistently didn’t reward their workers for doing good according to the org’s values, was serving no demand present in the market, and yet were competently executing some altruistic goal.
Right now I feel like I’m just saying a bunch of obvious things which you should definitely agree with, yet you believe we have a disagreement. I do not understand what you think I’m saying. Maybe you could try restating what I originally said in your own words?
We absolutely agree that incentives matter. Where I think we disagree is on how much they matter and how controllable they are. Especially for orgs whose goals are orthogonal or even contradictory with the common cultural and environmental incentives outside of the org.
I’m mostly reacting to your topic sentence
And wondering if ‘strikingly bad’ is relative to some EA or non-profit-driven org that does it well,or if ‘strikingly bad’ is just acknowledgement that it may not be possible to do well.
By strikingly bad I mean there are easy changes EA can make to make it’s sponsored orgs have better incentives, and it has too much confidence that the incentives in the orgs it sponsors favor doing good above doing bad, politics, not doing anything, etc.
For example, nobody in Anthropic gets paid more if they follow their RSP and less of they don’t. Changing this isn’t sufficient for me to feel happy with Anthropic, but its one example among many for which Anthropic could be better.
When I think of an Anthropic I feel happy with I think of a formally defined balance of powers type situation with strong & public whistleblower protection and post-whistleblower reform processes, them hiring engineers loyal to that process (rather than building AGI), and them diversifying the sources for which they trade, such that its in none of their source’s interest to manipulate them.
I also claim marginal movements toward this target are often good.
As I said in the original shortform, I also think incentives are not all or nothing. Worse incentives just mean you need more upstanding workers & leaders.