I think there’s a major bottleneck of really talented people, and that the field is not great at helping good but not amazing people be useful. These are pretty unfortunate dynamics in combination. But I do a lot of AI safety hiring, and even among people I hire there are big differences in ability which mean big differences in impact. If field building had not gotten those people into AI Safety, the world would be worse off
For each role, the impact different people would have in that role is heavy-tailed. So you can have loads of applicants (e.g. 100-to-1), but doubling your applicant pool would still yield big gains. The maths-y way to say this is that X is a distribution where E[max({X_i : i < 2n})]/E[max({X_i : i < n})] decays slowly in n, like Pareto or Lognormal. I guess this sucks for all (2n-1) people though.
Maybe the problem is partly the fellowships? Like, ideally, people would work on a cool project for a couple weeks (which is a commitment but exactly a blood oath) and if they aren’t great then they can go back to whatever they were doing before, or switch to roles which aren’t so heavy-tailed. But this would require fellowships to be more honest with their participants about their skills, which is unlikely to happen imo.
For each role, the impact different people would have in that role is heavy-tailed. So you can have loads of applicants (e.g. 100-to-1), but doubling your applicant pool would still yield big gains.
Whether this is true depends on how effectively you can filter your applicant pool, the marginal cost of processing additional applications, and the quality of the marginal applicants.
I think there’s a major bottleneck of really talented people, and that the field is not great at helping good but not amazing people be useful. These are pretty unfortunate dynamics in combination. But I do a lot of AI safety hiring, and even among people I hire there are big differences in ability which mean big differences in impact. If field building had not gotten those people into AI Safety, the world would be worse off
For each role, the impact different people would have in that role is heavy-tailed. So you can have loads of applicants (e.g. 100-to-1), but doubling your applicant pool would still yield big gains. The maths-y way to say this is that X is a distribution where E[max({X_i : i < 2n})]/E[max({X_i : i < n})] decays slowly in n, like Pareto or Lognormal. I guess this sucks for all (2n-1) people though.
Maybe the problem is partly the fellowships? Like, ideally, people would work on a cool project for a couple weeks (which is a commitment but exactly a blood oath) and if they aren’t great then they can go back to whatever they were doing before, or switch to roles which aren’t so heavy-tailed. But this would require fellowships to be more honest with their participants about their skills, which is unlikely to happen imo.
Whether this is true depends on how effectively you can filter your applicant pool, the marginal cost of processing additional applications, and the quality of the marginal applicants.
Yep.