Yeah! I think we share a similar vision, at least at the early stages where rogue AIs proliferate. I think the largest difference between the vision that Alvin proposed and what I wrote above is the consequences of proliferation—he assumes that various actors unite to put resources behind an aligned super intelligence that dominates all other rogue AI systems. In my vision this never happens and things stay permanently chaotic, leading to a shift in priorities from “fighting” AI destruction to being robust to it. The vision of the highly-robust bunker-state of the far future was the big thing I wanted to emphasize.
I remember Kokotajlo’s complaint about the optimistic 2027 timeline which just ended too soon. It means that we’d also need to consider that, say, by 2035, either someone will either create a superintelligence, aligned or not, or every human and AI will understand why they shouldn’t do it or can’t do it. What do you think will happen? A demo that ASI is uncreatable or unalignable? A decision to shut down AI research?
Yeah I share a similar intuition. It seems to me that the two steady states are either strong restrictions by some dominant force, or else proliferation of superintelligence into many distinct and diverse entities, all vying for their own interests
Yeah! I think we share a similar vision, at least at the early stages where rogue AIs proliferate. I think the largest difference between the vision that Alvin proposed and what I wrote above is the consequences of proliferation—he assumes that various actors unite to put resources behind an aligned super intelligence that dominates all other rogue AI systems. In my vision this never happens and things stay permanently chaotic, leading to a shift in priorities from “fighting” AI destruction to being robust to it. The vision of the highly-robust bunker-state of the far future was the big thing I wanted to emphasize.
I remember Kokotajlo’s complaint about the optimistic 2027 timeline which just ended too soon. It means that we’d also need to consider that, say, by 2035, either someone will either create a superintelligence, aligned or not, or every human and AI will understand why they shouldn’t do it or can’t do it. What do you think will happen? A demo that ASI is uncreatable or unalignable? A decision to shut down AI research?
Yeah I share a similar intuition. It seems to me that the two steady states are either strong restrictions by some dominant force, or else proliferation of superintelligence into many distinct and diverse entities, all vying for their own interests