Thanks for providing this! Could you say a little bit about what you were aiming for with these reviews? I find the collection of people involved somewhat surprising (although they’re all individually great) - particularly the lack of mainstream ML researchers.
Also, I hadn’t seen this framing before, so I enjoyed this section of Leopold’s response:
I could tell you about an agent in the world that is very misaligned (downright evil, proto-fascist, depending on who you ask), has the brainpower equivalent to 1.4 billion people, is in control of the second largest military in the world, and is probably power-seeking. This seems like a pretty worst-case version of the scenario outlined in the report. This agent is called China! Fwiw, I am quite worried about China. But I see the argument about AI as analogous to an argument about other misaligned agents. And my prior on other misaligned agents is that competition and correction is generally very good at keeping them in check.
Reviewers ended up on the list via different routes. A few we solicited specifically because we expected them to have relatively well-developed views that disagree with the report in one direction or another (e.g., more pessimistic, or more optimistic), and we wanted to understand the best objections in this respect. A few came from trying to get information about how generally thoughtful folks with different backgrounds react to the report. A few came from sending a note to GPI saying we were open to GPI folks providing reviews. And a few came via other miscellaneous routes. I’d definitely be interested to see more reviews from mainstream ML researchers, but understanding how ML researchers in particular react to the report wasn’t our priority here.
Thanks for providing this! Could you say a little bit about what you were aiming for with these reviews? I find the collection of people involved somewhat surprising (although they’re all individually great) - particularly the lack of mainstream ML researchers.
Also, I hadn’t seen this framing before, so I enjoyed this section of Leopold’s response:
Reviewers ended up on the list via different routes. A few we solicited specifically because we expected them to have relatively well-developed views that disagree with the report in one direction or another (e.g., more pessimistic, or more optimistic), and we wanted to understand the best objections in this respect. A few came from trying to get information about how generally thoughtful folks with different backgrounds react to the report. A few came from sending a note to GPI saying we were open to GPI folks providing reviews. And a few came via other miscellaneous routes. I’d definitely be interested to see more reviews from mainstream ML researchers, but understanding how ML researchers in particular react to the report wasn’t our priority here.