This is a similar idea to meta learning (e.g. MAML). It kicks the problem up a level: now the question is whether your AI system will generalize to the types of “distribution shifts” it has seen in the past.
Thanks. It’s similar in one sense, but (if I’m reading the paper right) a key difference is that in the MAML examples, the ordering of the meta-level and object level training is such that you still wind up optimizing hard for a particular goal. The idea here is that the two types of training function in opposition, as a control system of sorts, such that the meta-level training should make the model perform worse at the narrow type of task it was trained on.
That said, for sure, the types of distribution shift thing is an issue. It seems like this meta-level bias might be less bad than at the object level, but I have no idea.
This is a similar idea to meta learning (e.g. MAML). It kicks the problem up a level: now the question is whether your AI system will generalize to the types of “distribution shifts” it has seen in the past.
Thanks. It’s similar in one sense, but (if I’m reading the paper right) a key difference is that in the MAML examples, the ordering of the meta-level and object level training is such that you still wind up optimizing hard for a particular goal. The idea here is that the two types of training function in opposition, as a control system of sorts, such that the meta-level training should make the model perform worse at the narrow type of task it was trained on.
That said, for sure, the types of distribution shift thing is an issue. It seems like this meta-level bias might be less bad than at the object level, but I have no idea.