My proposed answer is: it means restricting or prohibiting the use of that collection of weights. If that collection of weights is particularly bad, then this directly minimises the harm. Given that it cost so much to create them, it also dis-incentivises the vendor from performing sets of weights which will facilitate crimes.
The biggest problem I see with this strategy is that neural nets are “fine tuned”, often with reinforcement learning from human feedback (RLHF). So the question becomes what region within the space of possible weights you are restricting. If you restrict those specific weights, what stops me from adding 0.0001 to one of the weights at random and getting a model that performs almost exactly the same that is no longer restricted? If you are restricting a region around those weights, how, technically, do you define that region? You would need to account for the many possible kinds of symmetry that give a model with exactly the same behaviour, and the semantic relationship between the space of possible weights and the behaviour of the model, which, despite the best efforts of mechanistic interpretability researchers, is still not well understood.
The nasty answer would be ‘all of it, back to the original training run, including all of the other descendants. Now start over.’.
The answer which actually relates to future consequence would require understanding (for example) multiplexing, the ability to reduce two AIs to some canonical form, and the ability to compare two canonical forms. Yes, we’re not there yet.
I wonder where “it’s the manufacturer’s responsibility to prove that it’s not substantially the same” would fit into our existing case-law of responsibility.
The biggest problem I see with this strategy is that neural nets are “fine tuned”, often with reinforcement learning from human feedback (RLHF). So the question becomes what region within the space of possible weights you are restricting. If you restrict those specific weights, what stops me from adding 0.0001 to one of the weights at random and getting a model that performs almost exactly the same that is no longer restricted? If you are restricting a region around those weights, how, technically, do you define that region? You would need to account for the many possible kinds of symmetry that give a model with exactly the same behaviour, and the semantic relationship between the space of possible weights and the behaviour of the model, which, despite the best efforts of mechanistic interpretability researchers, is still not well understood.
The nasty answer would be ‘all of it, back to the original training run, including all of the other descendants. Now start over.’.
The answer which actually relates to future consequence would require understanding (for example) multiplexing, the ability to reduce two AIs to some canonical form, and the ability to compare two canonical forms. Yes, we’re not there yet.
I wonder where “it’s the manufacturer’s responsibility to prove that it’s not substantially the same” would fit into our existing case-law of responsibility.