Yep, it’s a leap. It’s justified though IMO; we really do know so very little about these systems… I would be quite surprised if it turns out GPT-29 is a powerful agent with desires to influence the real world, but I wouldn’t be so surprised that I’d be willing to bet my eternal soul on it now. (Quantitatively I have something like 5% credence that it would be a powerful agent with desires to influence the real world.)
I am not sure your argument makes sense. Why think that its instincts and goals and whatnot refer only to what token to output in the domain of text? How is that different from saying “Whatever goals the coinrun agent has, they surely aren’t about anything in the game; instead they must be about which virtual buttons to press.” GPT is clearly capable of referring to and thinking about things in the real world; if it didn’t have a passable model of the real world it wouldn’t be able to predict text so accurately.
Yep, it’s a leap. It’s justified though IMO; we really do know so very little about these systems… I would be quite surprised if it turns out GPT-29 is a powerful agent with desires to influence the real world, but I wouldn’t be so surprised that I’d be willing to bet my eternal soul on it now. (Quantitatively I have something like 5% credence that it would be a powerful agent with desires to influence the real world.)
I am not sure your argument makes sense. Why think that its instincts and goals and whatnot refer only to what token to output in the domain of text? How is that different from saying “Whatever goals the coinrun agent has, they surely aren’t about anything in the game; instead they must be about which virtual buttons to press.” GPT is clearly capable of referring to and thinking about things in the real world; if it didn’t have a passable model of the real world it wouldn’t be able to predict text so accurately.