I’m certainly not arguing against red lines in general—red lines are great—if, and only if, they have force behind them. But our strategy now seems to be to draw lines that will go nowhere and can’t be enforced, making breaking them inevitable and irrelevant. There’s an old joke;
“The reservoir water level is so low we might run out completely.” “So what do you do when the water gets too low?” ”Oh, we just move the ‘Do Not Drink Below This Line’ sign five feet down again, and call it ‘restructuring the baseline’. ”
I do think that building a better norm would be valuable, but if we don’t see a path to doing so, continuing to draw new lines is at best unhelpful. At this point, we either need lines with teeth and political enforcement, or we need to stop drawing lines that aren’t going to matter, since that just reinforces the fact that we can’t and won’t do anything about it when they are broken.
I agree with most of this—red lines that aren’t respected are useless. However, stopping to draw red lines doesn’t solve any problems in my opinion—ignoring them or redrawing them is itself a signal. But I agree that what we need most is enforcement.
I think I disagree; every red line that is drawn and then ignored weakens the utility of drawing them. Now, I worry that if we end up in a situation where we need to say “dozens of people just died from an AI enabled chemical weapons attack,” it will be seen as yet another red line crossed, unsurprising, instead of a single obvious event that is way over any reasonable line.
Maybe we have a different understanding of the term “red line”. For me, it describes something that a human should not do, rather than an event that shouldn’t happen. So if someone releases an unsafe model, a red line is crossed when the model violates some defined safety specification, not when there’s a tragic event (which may or may not be a result of crossing the red line). However, I agree that in both cases there’s the danger of increasing numbness, so too much red line-drawing which then is simply ignored is indeed bad.
Meaningful red lines must be formally defined in a technical, near real-time enforcement system* with political enforcement backing—treated as hard-limit bans, not alarms. Non-technical red lines raise the will for such solutions, if they are not:
too much red line-drawing
EU AI Act style—complex regulatory red lines that exclude critical risk, are enforcement-intractable (or reactive) and serve as concern-stoppers.
Lines that are foreseeable unenforceable or carry a definite outcome if crossed (‘RSI will lead to loss-of-control’ vs ’RSI is an unacceptable loss-of-control risk″).
*TAIG in the time of Huawei/GLM-5 does throw sand & pebbles in the gears.
I’m certainly not arguing against red lines in general—red lines are great—if, and only if, they have force behind them. But our strategy now seems to be to draw lines that will go nowhere and can’t be enforced, making breaking them inevitable and irrelevant. There’s an old joke;
I do think that building a better norm would be valuable, but if we don’t see a path to doing so, continuing to draw new lines is at best unhelpful. At this point, we either need lines with teeth and political enforcement, or we need to stop drawing lines that aren’t going to matter, since that just reinforces the fact that we can’t and won’t do anything about it when they are broken.
I agree with most of this—red lines that aren’t respected are useless. However, stopping to draw red lines doesn’t solve any problems in my opinion—ignoring them or redrawing them is itself a signal. But I agree that what we need most is enforcement.
I think I disagree; every red line that is drawn and then ignored weakens the utility of drawing them. Now, I worry that if we end up in a situation where we need to say “dozens of people just died from an AI enabled chemical weapons attack,” it will be seen as yet another red line crossed, unsurprising, instead of a single obvious event that is way over any reasonable line.
Maybe we have a different understanding of the term “red line”. For me, it describes something that a human should not do, rather than an event that shouldn’t happen. So if someone releases an unsafe model, a red line is crossed when the model violates some defined safety specification, not when there’s a tragic event (which may or may not be a result of crossing the red line). However, I agree that in both cases there’s the danger of increasing numbness, so too much red line-drawing which then is simply ignored is indeed bad.
Meaningful red lines must be formally defined in a technical, near real-time enforcement system* with political enforcement backing—treated as hard-limit bans, not alarms. Non-technical red lines raise the will for such solutions, if they are not:
EU AI Act style—complex regulatory red lines that exclude critical risk, are enforcement-intractable (or reactive) and serve as concern-stoppers.
Lines that are foreseeable unenforceable or carry a definite outcome if crossed (‘RSI will lead to loss-of-control’ vs ’RSI is an unacceptable loss-of-control risk″).
*TAIG in the time of Huawei/GLM-5 does throw sand & pebbles in the gears.