I don’t know whether I’m an optimist or a doomer. I have two very specific responses to different parts of the situation:
the establishment of AI red lines
Ok. So, if:
a mixture of humans and LLMs together make a decision and carry it out, then
the act and/or some of its consequences is criminalizable,
then we have procedures (however imperfect) for allocating criminal liability among humans. What does it mean to allocate criminal liability to LLMs?
My proposed answer is: it means restricting or prohibiting the use of that collection of weights. If that collection of weights is particularly bad, then this directly minimises the harm. Given that it cost so much to create them, it also dis-incentivises the vendor from performing sets of weights which will facilitate crimes.
Does anyone want to work this up into a real proposal?
the key question is how to steer the future toward better outcome
I agree and, regardless of consistency with the proposal above, I like gentle steering early.
My imaginary plan looks like having a stable of mildly evil models, and having them have iterated conversations with a model under test, on the general subject of what to do next, or what to do next about the humans.
If the mildly evil model converts the model under test, that’s a black mark. If it’s the other way around, it’s hopeful indicative evidence.
My proposed answer is: it means restricting or prohibiting the use of that collection of weights. If that collection of weights is particularly bad, then this directly minimises the harm. Given that it cost so much to create them, it also dis-incentivises the vendor from performing sets of weights which will facilitate crimes.
The biggest problem I see with this strategy is that neural nets are “fine tuned”, often with reinforcement learning from human feedback (RLHF). So the question becomes what region within the space of possible weights you are restricting. If you restrict those specific weights, what stops me from adding 0.0001 to one of the weights at random and getting a model that performs almost exactly the same that is no longer restricted? If you are restricting a region around those weights, how, technically, do you define that region? You would need to account for the many possible kinds of symmetry that give a model with exactly the same behaviour, and the semantic relationship between the space of possible weights and the behaviour of the model, which, despite the best efforts of mechanistic interpretability researchers, is still not well understood.
The nasty answer would be ‘all of it, back to the original training run, including all of the other descendants. Now start over.’.
The answer which actually relates to future consequence would require understanding (for example) multiplexing, the ability to reduce two AIs to some canonical form, and the ability to compare two canonical forms. Yes, we’re not there yet.
I wonder where “it’s the manufacturer’s responsibility to prove that it’s not substantially the same” would fit into our existing case-law of responsibility.
I don’t know whether I’m an optimist or a doomer. I have two very specific responses to different parts of the situation:
Ok. So, if:
a mixture of humans and LLMs together make a decision and carry it out, then
the act and/or some of its consequences is criminalizable,
then we have procedures (however imperfect) for allocating criminal liability among humans. What does it mean to allocate criminal liability to LLMs?
My proposed answer is: it means restricting or prohibiting the use of that collection of weights. If that collection of weights is particularly bad, then this directly minimises the harm. Given that it cost so much to create them, it also dis-incentivises the vendor from performing sets of weights which will facilitate crimes.
Does anyone want to work this up into a real proposal?
I agree and, regardless of consistency with the proposal above, I like gentle steering early.
I think that there might be a black-box way of testing alignment using this effect:
https://www.astralcodexten.com/p/the-claude-bliss-attractor
where itterated interaction amplifies biasses.
My imaginary plan looks like having a stable of mildly evil models, and having them have iterated conversations with a model under test, on the general subject of what to do next, or what to do next about the humans.
If the mildly evil model converts the model under test, that’s a black mark. If it’s the other way around, it’s hopeful indicative evidence.
The biggest problem I see with this strategy is that neural nets are “fine tuned”, often with reinforcement learning from human feedback (RLHF). So the question becomes what region within the space of possible weights you are restricting. If you restrict those specific weights, what stops me from adding 0.0001 to one of the weights at random and getting a model that performs almost exactly the same that is no longer restricted? If you are restricting a region around those weights, how, technically, do you define that region? You would need to account for the many possible kinds of symmetry that give a model with exactly the same behaviour, and the semantic relationship between the space of possible weights and the behaviour of the model, which, despite the best efforts of mechanistic interpretability researchers, is still not well understood.
The nasty answer would be ‘all of it, back to the original training run, including all of the other descendants. Now start over.’.
The answer which actually relates to future consequence would require understanding (for example) multiplexing, the ability to reduce two AIs to some canonical form, and the ability to compare two canonical forms. Yes, we’re not there yet.
I wonder where “it’s the manufacturer’s responsibility to prove that it’s not substantially the same” would fit into our existing case-law of responsibility.