I remember, when you outlined your threat model, asking what you thought of MAI’s work. Your response made me feel depressed, and might have been the main reason I stopped being interested in their work.
Sorry, I hadn’t realized that I had been depressing twice when replying to your posts! My apologies!
It’s perfectly OK to model me as the person making the argument that ASI is just inherently risky, for Darwinian/economic/polical reasons. But my personal P(doom) was only about 30% the last time I did the math, because I actually see a bunch of potential off-roads. For example, if we see mass job losses more than a couple of years before ASI, I would actually expect a significant political backlash. If the public ever believes that mass permanent unemployment and/or actual human extinction are on the table, then they may be very unhappy. (They’re already unhappy in the abstract, but it isn’t driving their decision making.)
I remember, when you outlined your threat model, asking what you thought of MAI’s work. Your response made me feel depressed, and might have been the main reason I stopped being interested in their work.
Sorry, I hadn’t realized that I had been depressing twice when replying to your posts! My apologies!
It’s perfectly OK to model me as the person making the argument that ASI is just inherently risky, for Darwinian/economic/polical reasons. But my personal P(doom) was only about 30% the last time I did the math, because I actually see a bunch of potential off-roads. For example, if we see mass job losses more than a couple of years before ASI, I would actually expect a significant political backlash. If the public ever believes that mass permanent unemployment and/or actual human extinction are on the table, then they may be very unhappy. (They’re already unhappy in the abstract, but it isn’t driving their decision making.)