Online training infrastructure: For diffuse (non-concentrated) issues, training the AIs to behave better online (on the actual tasks we are using the AIs for rather than just on separate training data) could be very useful.
Quick note: under distribution shift, online approaches (including training, but also other kinds of validation or iteration) could be very important even for concentrated threats. Because of this, I’d maybe consider online training infrastructure to be higher on the list of software infrastructure interventions (not very confident, though).
I agree it’s also helpful for concentrated threats (or diffuse strategies which ultimatetly culminate in some concentrated failure, e.g. consistently looking more suspicious so that at a given FPR the TPR is lower).
Quick note: under distribution shift, online approaches (including training, but also other kinds of validation or iteration) could be very important even for concentrated threats. Because of this, I’d maybe consider online training infrastructure to be higher on the list of software infrastructure interventions (not very confident, though).
I agree it’s also helpful for concentrated threats (or diffuse strategies which ultimatetly culminate in some concentrated failure, e.g. consistently looking more suspicious so that at a given FPR the TPR is lower).