I think it’s fairly likely that the “devil is in the details” for AI safety. For example, some AI safety people like to talk about computer security, but I can’t think of any grand theories which have been useful there. Even for cryptography, we don’t have a proof that factoring large integers can’t be done efficiently, but this problem’s difficulty “in practice” still forms the basis of widely used cryptosystems. And the discipline devoted to studying how statistics can go wrong in practice seems to be called “applied statistics”, not “theoretical statistics”. So I think it’s plausibly valuable to have people concerned with safety at the cutting edge of AI development, thinking in a proactive way about the details of how the algorithms work and how they might go wrong.
I think it’s fairly likely that the “devil is in the details” for AI safety. For example, some AI safety people like to talk about computer security, but I can’t think of any grand theories which have been useful there. Even for cryptography, we don’t have a proof that factoring large integers can’t be done efficiently, but this problem’s difficulty “in practice” still forms the basis of widely used cryptosystems. And the discipline devoted to studying how statistics can go wrong in practice seems to be called “applied statistics”, not “theoretical statistics”. So I think it’s plausibly valuable to have people concerned with safety at the cutting edge of AI development, thinking in a proactive way about the details of how the algorithms work and how they might go wrong.