When Gemini randomly told an innocent user to go kill himself, it made the news, but this news didn’t really affect very much in the big picture.
It’s possible that relevant decision-makers don’t care that much about dramatic bad behaviours since the vibe is “oh yeah AI glitches up, oh well.”
It’s possible that relevant decision-makers do care more about what the top experts believe, and if the top experts are convinced that current models already want to kill you (but can’t), it may have an effect. Imagine if many top experts agree that “the lie detectors start blaring like crazy when the AI is explaining how it won’t kill all humans even if can get away with it.”
I’m not directly disagreeing with this post, I’m just saying there exists this possible world model where behavioural evidence isn’t much stronger (than other misalignment evidence).
@Knight Lee This is precisely one of the incidents I’ve seen (in Europe) Policy people refer to when arguing “why GenAI providers need to be held accountable” for misbehaviours like this. It is sad that this is example inspired regulatory actions in other jurisdictions, but not where the incident happen...
Oops. Maybe this kind of news does affect decision makers and I was wrong. I was just guessing that it had little effect, since… I’m not even sure why I thought so.
I did a Google search and it didn’t look like the kind of news that governments responded to.
No, you’re right! It is just a policy/ AI Safety advocacy argument, but one that does change minds and shape decisions. I guess it’s not as visible as it should be. Still, glad you brought this up!
When Gemini randomly told an innocent user to go kill himself, it made the news, but this news didn’t really affect very much in the big picture.
It’s possible that relevant decision-makers don’t care that much about dramatic bad behaviours since the vibe is “oh yeah AI glitches up, oh well.”
It’s possible that relevant decision-makers do care more about what the top experts believe, and if the top experts are convinced that current models already want to kill you (but can’t), it may have an effect. Imagine if many top experts agree that “the lie detectors start blaring like crazy when the AI is explaining how it won’t kill all humans even if can get away with it.”
I’m not directly disagreeing with this post, I’m just saying there exists this possible world model where behavioural evidence isn’t much stronger (than other misalignment evidence).
@Knight Lee This is precisely one of the incidents I’ve seen (in Europe) Policy people refer to when arguing “why GenAI providers need to be held accountable” for misbehaviours like this.
It is sad that this is example inspired regulatory actions in other jurisdictions, but not where the incident happen...
Oops. Maybe this kind of news does affect decision makers and I was wrong. I was just guessing that it had little effect, since… I’m not even sure why I thought so.
I did a Google search and it didn’t look like the kind of news that governments responded to.
No, you’re right! It is just a policy/ AI Safety advocacy argument, but one that does change minds and shape decisions. I guess it’s not as visible as it should be. Still, glad you brought this up!