have you tried unfreezing the inoculation adapter but lowering it’s LR, say by one magnitude? I would think that the IA hardly is ideal, especially throughout training where the task adapter changes stuff, and the task-adapter will try to compensate for this—as such, it seems more desirable to allow for the IA to still make slight changes, but be too slow to meaningfully learn the novel, desirable traits. [esp if it already ‘used up’ it’s ranks]
The vanilla IA is already pretty good at suppressing the undesired traits; the challenge is more about retaining the desired ones. I tried what you suggest in one setup a while ago. IIRC, unfortunately, there were little benefits in terms of suppression, at a significant cost in terms of retention.
Retention of the desired traits is hindered by them being learned conditional on the undesired ones. That’s why we use gates (for GIA and CGIA) that learn attenuations of the IA: to reduce such conditionalization.
cool work!
have you tried unfreezing the inoculation adapter but lowering it’s LR, say by one magnitude? I would think that the IA hardly is ideal, especially throughout training where the task adapter changes stuff, and the task-adapter will try to compensate for this—as such, it seems more desirable to allow for the IA to still make slight changes, but be too slow to meaningfully learn the novel, desirable traits. [esp if it already ‘used up’ it’s ranks]
The vanilla IA is already pretty good at suppressing the undesired traits; the challenge is more about retaining the desired ones. I tried what you suggest in one setup a while ago. IIRC, unfortunately, there were little benefits in terms of suppression, at a significant cost in terms of retention. Retention of the desired traits is hindered by them being learned conditional on the undesired ones. That’s why we use gates (for GIA and CGIA) that learn attenuations of the IA: to reduce such conditionalization.