Whoever first said about AI safety that it is “talent-constrained”, whom I don’t know, I wonder if they meant that the people doing AI safety are not good enough to get their task done, and it would be nice to cast a large net and somehow find even more competent people. Like they want the top 1⁄100′000′000 instead of the top 1⁄100′000.
This is true is we see AI safety as a technical project—power laws apply.
What I worry about is how this messages for AI safety as a political project. Sure, safety research is more important than “hurt feelings” but those feelings are a real input into political movements. If you tell someone they’re not “talented” enough to contribute to your movement’s goals and keep repeating that they’re talent constrained, that’s not a great way to coalition build.
Politics is the mind killer but it’s also the only path we have for a pause or slowdown right now. All the technical alignment in the world may not matter if AI safety convinces potential allies that they are untalented.
Maybe they think that, but I think it’s pretty harmful to think that? Those top 1⁄100,000 people can be working on other good things, and having a better time with their lives. To be clear I also don’t believe there are any problems that only the top 1⁄100,000,000 can solve, and I always felt like people who believe it come from a weird psychology. Even in theoretical physics (my academic field before I left) I think people are much more clear-eyed about the external factors that determine whether or not one can do good work beyond someone’s intrinsic competence/abilities/intelligence.
Whoever first said about AI safety that it is “talent-constrained”, whom I don’t know, I wonder if they meant that the people doing AI safety are not good enough to get their task done, and it would be nice to cast a large net and somehow find even more competent people. Like they want the top 1⁄100′000′000 instead of the top 1⁄100′000.
This is true is we see AI safety as a technical project—power laws apply.
What I worry about is how this messages for AI safety as a political project. Sure, safety research is more important than “hurt feelings” but those feelings are a real input into political movements. If you tell someone they’re not “talented” enough to contribute to your movement’s goals and keep repeating that they’re talent constrained, that’s not a great way to coalition build.
Politics is the mind killer but it’s also the only path we have for a pause or slowdown right now. All the technical alignment in the world may not matter if AI safety convinces potential allies that they are untalented.
Maybe they think that, but I think it’s pretty harmful to think that? Those top 1⁄100,000 people can be working on other good things, and having a better time with their lives. To be clear I also don’t believe there are any problems that only the top 1⁄100,000,000 can solve, and I always felt like people who believe it come from a weird psychology. Even in theoretical physics (my academic field before I left) I think people are much more clear-eyed about the external factors that determine whether or not one can do good work beyond someone’s intrinsic competence/abilities/intelligence.