I think it’s great to teach a course like this at good universities. I do think however, that the proximity to OpenAI comes with certain risk factors, from OpenAI’s official alignment blog: https://alignment.openai.com/hello-world/ ” We want to [..] develop and deploy [..] capable of recursive self-improvement (RSI)” This seems extremely dangerous to me, not on the scale we need to be a little careful, but on the scale of building mirror life bacteria or worse. Beyond, let’s research and more like, perhaps don’t do this. I worry that such concerns are not discusses in these courses and brushed aside against the “real risks” which are typically short term immediate harms that could reflect badly on these AI companies. Some people in academia are now launching workshops on recursive self-improvement: https://recursive-workshop.github.io
I would not go so far to say simply having proximity to OpenAI is problematic as long as the instructor manages to convey that their decision to associate with a leading AI company is controversial (or at least the more general point, that many reasonable people find what the AI companies do to be highly irresponsible). I somewhat trust that this is handled reasonably in this case. Overall, I think the worry here is less that people who might only be somewhat safety-pilled give courses like this, and more that people who are very safety pilled don’t have the opportunity to do so (due to selection effects in CS/ML academia).
I think it’s great to teach a course like this at good universities. I do think however, that the proximity to OpenAI comes with certain risk factors, from OpenAI’s official alignment blog: https://alignment.openai.com/hello-world/ ” We want to [..] develop and deploy [..] capable of recursive self-improvement (RSI)” This seems extremely dangerous to me, not on the scale we need to be a little careful, but on the scale of building mirror life bacteria or worse. Beyond, let’s research and more like, perhaps don’t do this. I worry that such concerns are not discusses in these courses and brushed aside against the “real risks” which are typically short term immediate harms that could reflect badly on these AI companies. Some people in academia are now launching workshops on recursive self-improvement: https://recursive-workshop.github.io
I would not go so far to say simply having proximity to OpenAI is problematic as long as the instructor manages to convey that their decision to associate with a leading AI company is controversial (or at least the more general point, that many reasonable people find what the AI companies do to be highly irresponsible). I somewhat trust that this is handled reasonably in this case.
Overall, I think the worry here is less that people who might only be somewhat safety-pilled give courses like this, and more that people who are very safety pilled don’t have the opportunity to do so (due to selection effects in CS/ML academia).