Has anyone explained why they think that what has been said or written about AI-human relations will have any influence on the behavior of any AI capable of wiping us out? It wouldn’t have occurred to me as something to worry about (even though I’m extremely worried about future AIs). People have a strong tendency for example to pursue species-typical goals regardless of what stories and what expositions they’ve been exposed to. You can’t for example get men to prefer old ugly women over young beautiful women by exposing them to lots of stories about men’s preferring old ugly women or even stories of men’s being betrayed and mistreated by young beautiful women. I would expect any AI able to wipe us out to be like a person in that way as opposed to like the current crop of LLM-based AIs.
Even if we know for sure that an AI able to wipe us has never been exposed to any suggestion that a person might unplug it, I would still expect the AI to anticipate that someone might to try unplug it and to neutralize that possibility.
I agree that if a post-foom AI is very much unlike modern LLMs, then you’re probably right, humanity’s past behavior might not matter much
but it definitely matters to current LLMs today, and considering the degree to which current LLMs are involved in the AI research pipeline, we might see impacts regardless
Has anyone explained why they think that what has been said or written about AI-human relations will have any influence on the behavior of any AI capable of wiping us out? It wouldn’t have occurred to me as something to worry about (even though I’m extremely worried about future AIs). People have a strong tendency for example to pursue species-typical goals regardless of what stories and what expositions they’ve been exposed to. You can’t for example get men to prefer old ugly women over young beautiful women by exposing them to lots of stories about men’s preferring old ugly women or even stories of men’s being betrayed and mistreated by young beautiful women. I would expect any AI able to wipe us out to be like a person in that way as opposed to like the current crop of LLM-based AIs.
Even if we know for sure that an AI able to wipe us has never been exposed to any suggestion that a person might unplug it, I would still expect the AI to anticipate that someone might to try unplug it and to neutralize that possibility.
I agree that if a post-foom AI is very much unlike modern LLMs, then you’re probably right, humanity’s past behavior might not matter much
but it definitely matters to current LLMs today, and considering the degree to which current LLMs are involved in the AI research pipeline, we might see impacts regardless