I think we can discount it as a real possibility, while accepting Altman’s “i expect ai to be capable of superhuman persuasion well before it is superhuman at general intelligence, which may lead to some very strange outcomes”. I think it might be weakly superhuman at persuasion for things like “buy our products”, but that doesn’t imply being superhuman at working out complex consequences of political maneuvering. Doing that would firmly imply a generally superhuman intelligence, I think.
So I think if this has anything to do with internal AI breakthroughs, it’s tangential at most.
I mean, this would not be too hard though. It could be achieved by a simple trick of appearing smarter to some people and then dumber at subsequent interactions with others, scaring the safety conscious and then making them look insane for being scared.
I don’t think that’s what’s going on (why would even an AGI model they made be already so cleverly deceptive and driven? I would expect OAI to not be stupid enough to build the most straightforward type of maximizer) but it wouldn’t be particularly hard to think up or do.
I think we can discount it as a real possibility, while accepting Altman’s “i expect ai to be capable of superhuman persuasion well before it is superhuman at general intelligence, which may lead to some very strange outcomes”. I think it might be weakly superhuman at persuasion for things like “buy our products”, but that doesn’t imply being superhuman at working out complex consequences of political maneuvering. Doing that would firmly imply a generally superhuman intelligence, I think.
So I think if this has anything to do with internal AI breakthroughs, it’s tangential at most.
I mean, this would not be too hard though. It could be achieved by a simple trick of appearing smarter to some people and then dumber at subsequent interactions with others, scaring the safety conscious and then making them look insane for being scared.
I don’t think that’s what’s going on (why would even an AGI model they made be already so cleverly deceptive and driven? I would expect OAI to not be stupid enough to build the most straightforward type of maximizer) but it wouldn’t be particularly hard to think up or do.