Deep forgetting and unlearning are under-researched. While there are many types of methods that can be used for them, there is very little work to practically apply them for scoping, especially in state-of-the-art models. There is also no work of which I am aware to study combinations and synergies between techniques. This type of research seems important, and it is very neglected and tractable, so I expect that excellent work could be done in the near future.
Unfortunately I don’t think academia will handle this by default. The current field of machine unlearning focuses on a narrow threat model where the goal is to eliminate the impact of individual training datapoints on the trained model. Here’s the NeurIPS 2023 Machine Unlearning Challenge task:
The challenge centers on the scenario in which an age predictor is built from face image data and, after training, a certain number of images must be forgotten to protect the privacy or rights of the individuals concerned.
But if hazardous knowledge can be pinpointed to individual training datapoints, perhaps you could simply remove those points from the dataset before training. The more difficult threat model involves removing hazardous knowledge that can be synthesized from many datapoints which are individually innocuous. For example, a model might learn to conduct cyberattacks or advise in the acquisition of biological weapons after being trained on textbooks about computer science and biology. It’s unclear the extent to which this kind of hazardous knowledge can be removed without harming standard capabilities, but most of the current field of machine unlearning is not even working on this more ambitious problem.
Though, for the threat model of ‘hazardous knowledge that can be synthesized from many datapoints which are individually innocuous’, this could still be a win if you remove the ‘hazardous knowledge [that] can be pinpointed to individual training datapoints’ and e.g. this forces the model to perform more explicit reasoning through e.g. CoT, which could be easier to monitor (also see these theoretical papers on the need for CoT for increased expressivity/certain types of problems).
Although the NeurIPS challenge and prior ML lit on forgetting and influence functions seem worth keeping on the radar because they’re still closely-related to challenges here.
Hopefully, this is getting / will get better given e.g. the NeurIPS 2023 Machine Unlearning Challenge
Unfortunately I don’t think academia will handle this by default. The current field of machine unlearning focuses on a narrow threat model where the goal is to eliminate the impact of individual training datapoints on the trained model. Here’s the NeurIPS 2023 Machine Unlearning Challenge task:
But if hazardous knowledge can be pinpointed to individual training datapoints, perhaps you could simply remove those points from the dataset before training. The more difficult threat model involves removing hazardous knowledge that can be synthesized from many datapoints which are individually innocuous. For example, a model might learn to conduct cyberattacks or advise in the acquisition of biological weapons after being trained on textbooks about computer science and biology. It’s unclear the extent to which this kind of hazardous knowledge can be removed without harming standard capabilities, but most of the current field of machine unlearning is not even working on this more ambitious problem.
+1
Though, for the threat model of ‘hazardous knowledge that can be synthesized from many datapoints which are individually innocuous’, this could still be a win if you remove the ‘hazardous knowledge [that] can be pinpointed to individual training datapoints’ and e.g. this forces the model to perform more explicit reasoning through e.g. CoT, which could be easier to monitor (also see these theoretical papers on the need for CoT for increased expressivity/certain types of problems).
+1
Although the NeurIPS challenge and prior ML lit on forgetting and influence functions seem worth keeping on the radar because they’re still closely-related to challenges here.
Thanks! I edited the post to add a link to this.