Curated! I think this is a fantastic contribution to the public discourse about AI control research. This really helped me think concretely about the development of AI and the likely causes of failure. I also really got a lot out of the visualization at the end of the “Failure to Generalize” section in terms of trying to understand why an AI’s cognition will be alien and hard to interpret. In my view there are already quite a lot of high-level alien forces running on humans (e.g. Moloch), and there will be high-level alien forces running on the simulated society in the AI’s mind.
I am glad that there’s a high-quality case for and against this line of research, it makes me feel positive about the state of discourse on this subject.
(Meta note: This curation notice accidentally went up 3 days after the post was curated.)
Curated! I think this is a fantastic contribution to the public discourse about AI control research. This really helped me think concretely about the development of AI and the likely causes of failure. I also really got a lot out of the visualization at the end of the “Failure to Generalize” section in terms of trying to understand why an AI’s cognition will be alien and hard to interpret. In my view there are already quite a lot of high-level alien forces running on humans (e.g. Moloch), and there will be high-level alien forces running on the simulated society in the AI’s mind.
I am glad that there’s a high-quality case for and against this line of research, it makes me feel positive about the state of discourse on this subject.
(Meta note: This curation notice accidentally went up 3 days after the post was curated.)