This might depend in part on what kind of odds you give places like Meta, OpenAI, and whether you think that they’re fundamentally doomed because of the structure of the problem.
If you think that OpenAI is absolutely doomed at how they proceed—marginal improvements make no difference—adding conscientious people makes no difference, so time of conscientious people is better spent anywhere else.
If you think that OpenAI stands a good chance of figuring things out—marginal improvements could make a big difference—then the kind of reasoning you point to is more persuasive.
I think Soares thinks we’re in the first world, so his prescriptions makes sense for that. (Probably also for deontological considerations, but again, these also look different in different worlds.)
This might depend in part on what kind of odds you give places like Meta, OpenAI, and whether you think that they’re fundamentally doomed because of the structure of the problem.
If you think that OpenAI is absolutely doomed at how they proceed—marginal improvements make no difference—adding conscientious people makes no difference, so time of conscientious people is better spent anywhere else.
If you think that OpenAI stands a good chance of figuring things out—marginal improvements could make a big difference—then the kind of reasoning you point to is more persuasive.
I think Soares thinks we’re in the first world, so his prescriptions makes sense for that. (Probably also for deontological considerations, but again, these also look different in different worlds.)