I’m not sure I have a most-preferred future—I guess it would be everyone becoming aware of AI risk and the U.N. with US and China at the helm co-ordinating a joint pause and eventual AGI project. But people who point out this is super dangerous itself due to power centralization are correct, and on top of that I think it’s possible that instead of co-ordination, everyone becoming aware of AI risk leads to racing, sabotage, and other drastic actions.
The current situation where the leading labs are moderately risk aware and the rest of the world doesn’t seem that freaked out, while still being freaked out enough to potentially apply some brakes and regulations, is pretty good I think.
Your preferred future seems both nearly-impossible, not that good if it happens, and if we fall short of total victory on your goal then it comes out strongly negative—if we shame AI researchers and only some people worried about AI risk quit, and most people not worried about it stay, then all we’ve done is slow things down 1% and decreased the awareness of the field by 10%. I agree that more time to adjust and develop things like algorithmic liability would be very helpful though, I just think the price of buying that extra time through researcher-exit is absurdly high.
Well, a version of the same plan already basically worked: discouraging nuclear power and encouraging renewables instead. We need the same kind of society-wide push against AI and for intelligence amplification / human-complementary tech. Not just discouraging a subset of researchers, but also creating regulatory hurdles, convincing the public that AI is harmful, and so on. I see nothing impossible about this.
The other stuff is fine, I’m making the narrow point that discouraging the subset of researchers who care about AI risk from working in the field is bad.
I’m not sure I have a most-preferred future—I guess it would be everyone becoming aware of AI risk and the U.N. with US and China at the helm co-ordinating a joint pause and eventual AGI project. But people who point out this is super dangerous itself due to power centralization are correct, and on top of that I think it’s possible that instead of co-ordination, everyone becoming aware of AI risk leads to racing, sabotage, and other drastic actions.
The current situation where the leading labs are moderately risk aware and the rest of the world doesn’t seem that freaked out, while still being freaked out enough to potentially apply some brakes and regulations, is pretty good I think.
Your preferred future seems both nearly-impossible, not that good if it happens, and if we fall short of total victory on your goal then it comes out strongly negative—if we shame AI researchers and only some people worried about AI risk quit, and most people not worried about it stay, then all we’ve done is slow things down 1% and decreased the awareness of the field by 10%. I agree that more time to adjust and develop things like algorithmic liability would be very helpful though, I just think the price of buying that extra time through researcher-exit is absurdly high.
Well, a version of the same plan already basically worked: discouraging nuclear power and encouraging renewables instead. We need the same kind of society-wide push against AI and for intelligence amplification / human-complementary tech. Not just discouraging a subset of researchers, but also creating regulatory hurdles, convincing the public that AI is harmful, and so on. I see nothing impossible about this.
The other stuff is fine, I’m making the narrow point that discouraging the subset of researchers who care about AI risk from working in the field is bad.