I think these intuitions exist and explain some level of “AI is just like other software” with models today. But the biggest psychological hangup I encounter most in people is more like a vague but very strong intuition like “humans will always control the planet, nothing more intelligent than us can exist or would influence the world more than we do.” i.e. Even if we could architect an ASI like other software most wouldn’t infer that this has drastic implications for the world order/political economy
I think these intuitions exist and explain some level of “AI is just like other software” with models today. But the biggest psychological hangup I encounter most in people is more like a vague but very strong intuition like “humans will always control the planet, nothing more intelligent than us can exist or would influence the world more than we do.” i.e. Even if we could architect an ASI like other software most wouldn’t infer that this has drastic implications for the world order/political economy