I think this is accurate. Relatedly (but not as a retort or anything, I’m curious what people think) is it bad or immoral (or something along those lines) for individuals who are passionate (via intellectual interest, etc...) about specific AI safety verticals to pursue them?
For example, there are plenty of people who like science of deep learning/interpretability/vaguely things that involve understanding how DNNs “work”. It’s probably pretty easy to argue that this is unlikely to be the highest impact to reduce existential risk. Even if I’m wrong in thinking that this is easy to argue, then you could probably argue it for a lot of specific projects people are engaging in. However, I think it’s an easy to tell yourself that you are doing such research because of and to mitigate existential risk.
In such a (hypothetical) set of scenarios, let’s assume these individuals would be actively unhappy and slightly worse than their peers doing direct/straightforward DIP-like policy/politics/outreach work. So basically these hypothetical people need to pick between something useful and something fun.
So the question becomes what should these people do?
More broadly, I think doing straightforward work that is more likely to solve the problems of our society just tends not to be what people are individually interested in doing and thus we see “The Spectre” and its cousins in AI safety and probably other fields too (my generalization here is to situations where people do X because of Y but claim its because of Z where Y is provides a bit more individual utility and Z provides a bit more societal utility).
EDIT: I don’t actually know much about Control AI and I see some people are contesting some assumptions here (such as the fact that Control AI has been successful). I don’t have time to read this. I think broadly that doesn’t actually matter to my post here, because my post is (1) not about Control AI in particular, but instead about cases in AIS where you have X, Y, and Z as described ^, (2) on priors this 4-step plan stuff seems kind of self-evidently what needs to be done to get a lot of needed actions to occur (i.e. for regulation, etc...).
I think this is accurate. Relatedly (but not as a retort or anything, I’m curious what people think) is it bad or immoral (or something along those lines) for individuals who are passionate (via intellectual interest, etc...) about specific AI safety verticals to pursue them?
For example, there are plenty of people who like science of deep learning/interpretability/vaguely things that involve understanding how DNNs “work”. It’s probably pretty easy to argue that this is unlikely to be the highest impact to reduce existential risk. Even if I’m wrong in thinking that this is easy to argue, then you could probably argue it for a lot of specific projects people are engaging in. However, I think it’s an easy to tell yourself that you are doing such research because of and to mitigate existential risk.
In such a (hypothetical) set of scenarios, let’s assume these individuals would be actively unhappy and slightly worse than their peers doing direct/straightforward DIP-like policy/politics/outreach work. So basically these hypothetical people need to pick between something useful and something fun.
So the question becomes what should these people do?
More broadly, I think doing straightforward work that is more likely to solve the problems of our society just tends not to be what people are individually interested in doing and thus we see “The Spectre” and its cousins in AI safety and probably other fields too (my generalization here is to situations where people do X because of Y but claim its because of Z where Y is provides a bit more individual utility and Z provides a bit more societal utility).
EDIT: I don’t actually know much about Control AI and I see some people are contesting some assumptions here (such as the fact that Control AI has been successful). I don’t have time to read this. I think broadly that doesn’t actually matter to my post here, because my post is (1) not about Control AI in particular, but instead about cases in AIS where you have X, Y, and Z as described ^, (2) on priors this 4-step plan stuff seems kind of self-evidently what needs to be done to get a lot of needed actions to occur (i.e. for regulation, etc...).