Ryan disagree-reacted to the bold part of this sentence in my comment above and I’m not sure why: “This tweet predicts two objections to this story that align with my first and third bullet point (common objections) above.”
This seems pretty unimportant to gain clarity on, but I’ll explain my original sentence more clearly anyway:
For reference, my third bullet point was the common objection: “How would humanity fail to notice this and/or stop this?”
To my mind, someone objecting that the story is unrealistic because “there’s no reason why OpenAI would ever let the model do its thinking steps in opaque vectors instead of written out in English” (as stated in the tweet) isan objection of the form “humanity wouldn’t fail to stop AI from sneakily engaging in power-seeking behavior by thinking in opaque vectors.” Like it’s a “sure, AI could takeover if humanity were dumb like that, but there’s no way OpenAI would be dump like that.”
It seems like Ryan was disagreeing with this with his emoji, but maybe I misunderstood it.
In the context of “can the AIs takeover?”, I was trying to point to the rogue AI intepretation. As in, even if the AIs were rogue and had a rogue internal deployment inside the frontier AI company, how do they end up with actual hard power. For catching already rogue AIs and stopping them, opaque vector reasoning doesn’t make much of a diffence.
Thanks for the clarification. My conclusion is that I think your emoji was meant to signal disagreement with the claim that ‘opaque vector reasoning makes a difference’ rather than a thing I believe.
I had rogue AIs in mind as well, and I’ll take your word on “for catching already rogue AIs and stopping them, opaque vector reasoning doesn’t make much of a difference”.
Ryan disagree-reacted to the bold part of this sentence in my comment above and I’m not sure why: “This tweet predicts two objections to this story that align with my first and third bullet point (common objections) above.”
This seems pretty unimportant to gain clarity on, but I’ll explain my original sentence more clearly anyway:
For reference, my third bullet point was the common objection: “How would humanity fail to notice this and/or stop this?”
To my mind, someone objecting that the story is unrealistic because “there’s no reason why OpenAI would ever let the model do its thinking steps in opaque vectors instead of written out in English” (as stated in the tweet) is an objection of the form “humanity wouldn’t fail to stop AI from sneakily engaging in power-seeking behavior by thinking in opaque vectors.” Like it’s a “sure, AI could takeover if humanity were dumb like that, but there’s no way OpenAI would be dump like that.”
It seems like Ryan was disagreeing with this with his emoji, but maybe I misunderstood it.
There are two interpretations you might have for that third bullet:
Can we stop rogue AIs? (Which are operating without human supervision.)
Can we stop AIs deployed in their intended context?
(See also here.)
In the context of “can the AIs takeover?”, I was trying to point to the rogue AI intepretation. As in, even if the AIs were rogue and had a rogue internal deployment inside the frontier AI company, how do they end up with actual hard power. For catching already rogue AIs and stopping them, opaque vector reasoning doesn’t make much of a diffence.
Thanks for the clarification. My conclusion is that I think your emoji was meant to signal disagreement with the claim that ‘opaque vector reasoning makes a difference’ rather than a thing I believe.
I had rogue AIs in mind as well, and I’ll take your word on “for catching already rogue AIs and stopping them, opaque vector reasoning doesn’t make much of a difference”.
I doubt that person was thinking about the opaque vector reasoning making it harder to catch the rogue AIs.