Probably the most important thing to do is slow down real AGI progress. This seems to be largely a social and political problem: getting researchers to not want to make AGI, and getting governments to outlaw it.
The second most important thing to do is to have a longer-term game plan. My bet is to increase human intelligence. If we have more collective brainpower, we’ll be better able to navigate the AGI transition—e.g. by figuring out how to make AGI safely and beneficially, or maybe by figuring out how to decide (as a species) to not make AGI, on a longer-term and more reliable basis. Also, the general prospect of increasing human capacity to solve problems might, I hope, offer a compelling alternative to developing AGI: we’ll have radical abundance, and there is a much smaller risk of power collapsing into one extremely powerful entity.
Probably the most important thing to do is slow down real AGI progress. This seems to be largely a social and political problem: getting researchers to not want to make AGI, and getting governments to outlaw it.
The second most important thing to do is to have a longer-term game plan. My bet is to increase human intelligence. If we have more collective brainpower, we’ll be better able to navigate the AGI transition—e.g. by figuring out how to make AGI safely and beneficially, or maybe by figuring out how to decide (as a species) to not make AGI, on a longer-term and more reliable basis. Also, the general prospect of increasing human capacity to solve problems might, I hope, offer a compelling alternative to developing AGI: we’ll have radical abundance, and there is a much smaller risk of power collapsing into one extremely powerful entity.
I’ve written about human intelligence amplification in general as a technical problem in Overview of strong human intelligence amplification methods, and I’ve written in great detail about the technical problem of the most promising route in Methods for strong human germline engineering. I’d be happy to make more concrete recommendations if you’re interested.