AI is a poorly understood, speculative field, so people with different intuitions come to different conclusions. If your intuition is that AI is inherently dangerous, it makes sense to seriously investigate the AI safety questions, whereas if you don’t, you would focus on the potential benefits and want to accelerate.
I remember when first reading about paperclip maximizer AIs I thought it was stupid, as in anything superintelligent would be capable of understanding instructions. I think this intuition hints at the stronger reason to be more optimistic about AI, which is opposition to the orthogonality thesis.
alexfeghhi
Karma: 2
I agree, but from a different angle.
There are several ways in which communicating probabilities offers useful information, and I don’t see p(doom) in any of them.
For one, there are true probabilities, such as the probability of a radioactive atom decaying by it’s half life. This satisfies the frequentist interpretation, where one could re-run the radioactive decay experiment infinitely and observe the true probabilities over time. This does not apply to p(doom); it is unclear how quantum uncertainty could lead to noticeably different outcomes were we to re-run the AI doom experiment. Instead, we lack the information to correctly predict the future.
This does not mean probabilities outside of quantum mechanics are useless. We can construct probabilistic models, such as how Nate Silver, an election forecaster, might add uncertainty to polls and simulate election outcomes. The issue is on what basis to construct a probabilistic model of AI doom. Probabilistic forecasters use base rates and update on event specific information. What are the base rates for an unprecedented, poorly understood, civilization altering technology, causing doom?
Finally, most people do not specify their p(doom) along with a probabilistic model. This is therefore an implicit appeal to authority. The ability to trust authorities depends on the subject. In AI, predictions on timelines and methods have historically been incorrect. It would be unusual if despite experts being unable to predict the how and when of AI, they could predict the impacts.
My suggestion for AI safety advocates is to embrace the unpredictability of the situation. If we really don’t understand AI that well, we also don’t know what its impacts will be and in particular, if it will be safe. Given that historically the approach with technology has been to push forward and retroactively assess the negative impacts, we run the serious risk of being insufficiently cautious.