Replying to David Hornbein.
Thank you for this comment, this was basically my view as well. I think the employees of OpenAI are simply excited about AGI, have committed their lives working long hours to make it a reality and believe AGI would be good for humanity and also good for them personally. My view is that they are very emotionally invested in building AGI and stopping all that progress for reasons that feel speculative, theoretical and not very tangible feels painful.
Not that I would agree with that, assuming this is correct.
I’m wondering what Nick Bostrom’s p(doom) currently is, given the subject of this book. He said 9 years ago in his lecture on his book Superintelligence “less than 50% risk of doom”. In this interview 4 months ago he said that it’s good there has been more focus on risks in recent times, but there’s still slightly less focus on the risks than what is optimal, but he wants to focus on the upsides because he fears we might “overshoot” and not build AGI at all which would be tragic in his opinion. So it seems he thinks the risk is less than it used to be because of this public awareness of the risks.