Yeah! I think we share a similar vision, at least at the early stages where rogue AIs proliferate. I think the largest difference between the vision that Alvin proposed and what I wrote above is the consequences of proliferation—he assumes that various actors unite to put resources behind an aligned super intelligence that dominates all other rogue AI systems. In my vision this never happens and things stay permanently chaotic, leading to a shift in priorities from “fighting” AI destruction to being robust to it. The vision of the highly-robust bunker-state of the far future was the big thing I wanted to emphasize.
I remember Kokotajlo’s complaint about the optimistic 2027 timeline which just ended too soon. It means that we’d also need to consider that, say, by 2035, either someone will either create a superintelligence, aligned or not, or every human and AI will understand why they shouldn’t do it or can’t do it. What do you think will happen? A demo that ASI is uncreatable or unalignable? A decision to shut down AI research?
Yeah I share a similar intuition. It seems to me that the two steady states are either strong restrictions by some dominant force, or else proliferation of superintelligence into many distinct and diverse entities, all vying for their own interests
So you have essentially rediscovered the concerns of @Alvin Ånestrand, who proposed to include rogue replication into the AI-2027 forecast back on May 30. I understand why, say, my idea to co-deploy many AIs was unnoticed until it was rediscovered by Cleo Nardo, but why did people miss Alvin Anestrand’s scenario?
Yeah! I think we share a similar vision, at least at the early stages where rogue AIs proliferate. I think the largest difference between the vision that Alvin proposed and what I wrote above is the consequences of proliferation—he assumes that various actors unite to put resources behind an aligned super intelligence that dominates all other rogue AI systems. In my vision this never happens and things stay permanently chaotic, leading to a shift in priorities from “fighting” AI destruction to being robust to it. The vision of the highly-robust bunker-state of the far future was the big thing I wanted to emphasize.
I remember Kokotajlo’s complaint about the optimistic 2027 timeline which just ended too soon. It means that we’d also need to consider that, say, by 2035, either someone will either create a superintelligence, aligned or not, or every human and AI will understand why they shouldn’t do it or can’t do it. What do you think will happen? A demo that ASI is uncreatable or unalignable? A decision to shut down AI research?
Yeah I share a similar intuition. It seems to me that the two steady states are either strong restrictions by some dominant force, or else proliferation of superintelligence into many distinct and diverse entities, all vying for their own interests