Neat idea. One thing I’d watch out for is that “subset of the neurons” might not be the right ontology for a conceptually-”small” change. E.g. in the Rome paper, they made low-rank updates rather than work with individual neurons. So bear in mind that figuring out the ontology through which to view the network’s internals may itself be part of the problem.
Neat idea. One thing I’d watch out for is that “subset of the neurons” might not be the right ontology for a conceptually-”small” change. E.g. in the Rome paper, they made low-rank updates rather than work with individual neurons. So bear in mind that figuring out the ontology through which to view the network’s internals may itself be part of the problem.