AI safety people often worry about their work having capabilities externalities. I feel like the inverse problem is often a much bigger deal: the stuff you do might be done later by the capabilities/product people in the course of their work, meaning your work isn’t counterfactual! I think people should try to distinguish these concerns more clearly.
(See also Bronson Schoen’s similar post three hours earlier. I wrote this shortform in Redwood slack a few weeks ago and am just posting it now. Obviously this isn’t a very original take.)
I agree one should focus on counterfactual impact, but simply because your work would eventually be done anyway does not mean the counterfactual impact is identical.
Specific research being done earlier may shorten timelines and/or encourage others to do more work on the same topic.
I feel like something stronger needs to be said: If this is the case, then you’re probably working on something that the capabilities/product people will need for their capabilities/product work, and therefore plausibly just doing for them a bit of their work, which is pushing the world towards x-risk.
It might be worth it anyway, because maybe it’s better for the world if this specific part of the work that you’re doing gets done earlier, relative to the other parts of the work. But, eh, IDK, seems sus.
AI safety people often worry about their work having capabilities externalities. I feel like the inverse problem is often a much bigger deal: the stuff you do might be done later by the capabilities/product people in the course of their work, meaning your work isn’t counterfactual! I think people should try to distinguish these concerns more clearly.
(See also Bronson Schoen’s similar post three hours earlier. I wrote this shortform in Redwood slack a few weeks ago and am just posting it now. Obviously this isn’t a very original take.)
I agree one should focus on counterfactual impact, but simply because your work would eventually be done anyway does not mean the counterfactual impact is identical.
Specific research being done earlier may shorten timelines and/or encourage others to do more work on the same topic.
I feel like something stronger needs to be said: If this is the case, then you’re probably working on something that the capabilities/product people will need for their capabilities/product work, and therefore plausibly just doing for them a bit of their work, which is pushing the world towards x-risk.
It might be worth it anyway, because maybe it’s better for the world if this specific part of the work that you’re doing gets done earlier, relative to the other parts of the work. But, eh, IDK, seems sus.
Wei Dai’s point on legible vs illegible problems is also related.
This is why I stopped working on cyber.
“you overestimate yourself, sir”