I think the best reason for hope here, is the cosmopolitanism of technical communication. I mentioned a version of this a few years ago. One cannot control the politics and culture of all the different contenders in the AI race, but they all pay attention to (and many contribute to) various public technical literatures, and so one may hope that crucial ideas regarding what is risky and what is safe, if published, will travel beyond the particular AI “research culture” that originates them.
I imagined the exact reply falling neatly into the many programs of asia (India, Singapore, Taiwan) where there isnt effective rach currently but there is high cultural compability and an open scientific culture. On the cases of China and Israel though the problem is both their state and private initiatives are more “hostile” to any sort of outsider idea coming from any place in the world which dictates that they should intentionally neuter their progress unless they have got a domestic legal reason to do so. We can hope that Alignment is such an important topic that it will get thorough somehow but considering how little reach it already has in an “open” enviroment that has been primed for taking near future issues seriously I dont see any dignified death scenario where atleast all current major AI companies have agreed their first priority is general human alignment.
Regarding AI research programs that are closed and are pushing capabilities ahead as fast as possible, I see two reasons for hope.
First: if they are smart enough to genuinely have a chance of creating superhuman AI, hopefully they are also smart enough to understand that superhuman AI could have its own agenda, and has the potential to be a threat from within, rivaling any external threats that may be motivating the researchers.
Second: as AI advances, AI itself can contribute to alignment theory. The technology itself therefore has some possibility of improving the strategic wisdom of any group trying to develop it.
I think the best reason for hope here, is the cosmopolitanism of technical communication. I mentioned a version of this a few years ago. One cannot control the politics and culture of all the different contenders in the AI race, but they all pay attention to (and many contribute to) various public technical literatures, and so one may hope that crucial ideas regarding what is risky and what is safe, if published, will travel beyond the particular AI “research culture” that originates them.
I imagined the exact reply falling neatly into the many programs of asia (India, Singapore, Taiwan) where there isnt effective rach currently but there is high cultural compability and an open scientific culture. On the cases of China and Israel though the problem is both their state and private initiatives are more “hostile” to any sort of outsider idea coming from any place in the world which dictates that they should intentionally neuter their progress unless they have got a domestic legal reason to do so. We can hope that Alignment is such an important topic that it will get thorough somehow but considering how little reach it already has in an “open” enviroment that has been primed for taking near future issues seriously I dont see any dignified death scenario where atleast all current major AI companies have agreed their first priority is general human alignment.
Regarding AI research programs that are closed and are pushing capabilities ahead as fast as possible, I see two reasons for hope.
First: if they are smart enough to genuinely have a chance of creating superhuman AI, hopefully they are also smart enough to understand that superhuman AI could have its own agenda, and has the potential to be a threat from within, rivaling any external threats that may be motivating the researchers.
Second: as AI advances, AI itself can contribute to alignment theory. The technology itself therefore has some possibility of improving the strategic wisdom of any group trying to develop it.