Note: This post was written in February 2019 while at the Governance of AI Programme, within the Future of Humanity Institute. I’m publishing it as it stood in February, despite significant flaws, since I’m starting a new job and anticipate I won’t have time to update it. I thank Markus Anderljung, Max Daniel, Jeffrey Ding, Eric Drexler, Carrick Flynn, Richard Ngo, Cullen O’Keefe, Stefan Schubert, Rohin Shah, Toby Shevlane, Matt van der Merwe and Remco Zwetsloot for help with previous versions of this document. Ben Garfinkel was especially generous with his time and many of the ideas in this document were originally his.
I guess this may have been one of those Google docs that people had a lot of private discussions in. This makes me rather discouraged from commenting, knowing that anything I write may have been extensively discussed already and the author just didn’t have time to or didn’t feel like incorporating those comments/viewpoints into the published document. (Some of the open questions listed seem to have fairly obvious answers. Did no one suggest such answers to the author? Or were they found wanting in some way?) Also it seems like the author is not here to participate in a public discussion, or may not have the time to do so (given his new job situation).
However I did write a bunch of comments under one of your posts (you = ricraz = Richard Ngo, if I remember correctly), which appears to cover roughly the same topic (shifts in AI risk arguments over time), and those comments may also be somewhat relevant here. Beyond that, I wonder if you could summarize what is new or different in this document compared to yours, and whether you think there’s anything in it that would be especially valuable to have a public discussion about (even absent participation of the author).
I guess this may have been one of those Google docs that people had a lot of private discussions in. This makes me rather discouraged from commenting, knowing that anything I write may have been extensively discussed already and the author just didn’t have time to or didn’t feel like incorporating those comments/viewpoints into the published document. (Some of the open questions listed seem to have fairly obvious answers. Did no one suggest such answers to the author? Or were they found wanting in some way?) Also it seems like the author is not here to participate in a public discussion, or may not have the time to do so (given his new job situation).
However I did write a bunch of comments under one of your posts (you = ricraz = Richard Ngo, if I remember correctly), which appears to cover roughly the same topic (shifts in AI risk arguments over time), and those comments may also be somewhat relevant here. Beyond that, I wonder if you could summarize what is new or different in this document compared to yours, and whether you think there’s anything in it that would be especially valuable to have a public discussion about (even absent participation of the author).