Since so many people here (myself included) are either working to reduce AI risk or would love to enter the field, it seems worthwhile to ask what are the best arguments against doing so. This question is intended to focus on existential/catastrophic risks and not things like technological unemployment and bias in machine learning algorithms.
[Question] Best arguments against worrying about AI risk?
Note that “best arguments against worrying about AI risk” and “best arguments against entering the AI risk field” are distinct issues. E.g. suppose that AI risk was the #1 thing we should be worrying about and biorisk was the #2 thing we should be worrying about. Then someone who already had a strong interest in biology might be better off entering the biorisk field because of the higher personal fit:
… the most successful people in a field account for a disproportionately large fraction of the impact. [...] Personal fit is like a multiplier of everything else, and this means it’s probably more important than the other three factors [of career capital, altruistic impact, and supportive job conditions]. So, we’d never recommend taking a “high impact” job that you’d be bad at.
(Made this a question on the author’s request)
You are risking to oversaturate the AGI research market relative to other life-changing potential technologies, such as engineered life extension or organizational mechanism design.
Such oversaturation may result in two or more internally consistent but poorly compatible AI Safety theories (e.g. distillation-amplification vs. agent foundation), which will end up in two different friendly AI and the the war for correct friendliness will start.
If we have two distinct AI safety plans, the researchers are sensible to have a big discussion on which is better and only turn that one on. If not, and neither AI is fatally flawed, I would expect them to cooperate, they have very similar goals and neither wants war.
Historically, it didn’t work. A most bitter conflicts were between two main branches of, say, Islam, - Shia and Sunni, or between different socialists groups. We could hope though that rationalists are better in cooperation.
Distillation-amplification, if it works, should only start a war if the amplified human would have wanted that. Agent foundations theorists afaic predict that the first mover in AI has enough strategic advantage that there’ll be nothing worthy of the word war.
Ok, so should amplified human try to turn off first the agent-foundation-based project which is going to turn off this human if complete?
If the amplified human could take over the world but hasn’t because he’s not evil, and predicts that this other AI system would do such evil, yes.
It’s plausible, though, that the decision theory used by the new AI would tell it to act predictably non-evilly, in order to make the amplified human see this coming and not destroy the new AI before it’s turned on.
Note that this amplified human has thereby already taken over the world in all but name.
There are two common arguments I hear against spending significant personal effort on the topic of AI risk. I don’t necessarily endorse either of these (though #1a is my biggest fear on the topic—there’s not much about prevention of AI divergence that isn’t pretty nightmarish when applied to prevention of human value divergence).
1) There’s no coherent description of how this work actually addresses any actual risk. I don’t see much reason to believe that working on AI risk actually reduces AI risk. Working on decision theory and moral philosophy may be useful for other things, of course.
2) On the margin (of your effort/results), there are topics you can make more personal impact on and have a larger overall impact on the future.
[ edited to fix formatting ]
Your comment is malformatted.
Some outside view arguments:
Forecasting is hard, and experts disagree with the AI risk community (though this is less true now, and may be a difference in values rather than beliefs, i.e. longtermism vs. common-sense morality)
Past doomsday predictions have all not materialized. (Anthropics might throw a wrench in this, but my impression is that predictions of non-existential catastrophes have been made but haven’t happened.) Even if you think you should focus solely on x-risk, this suggests you should focus on the ones that a large group of people agree are x-risks.
And some inside view arguments:
The underlying causes of x-risk scenarios also lead to problems before superintelligence, eg. reward hacking. We’ll notice these problems when they occur and correct them. (Note that they won’t be corrected in the near term because the failures are so inconsequential that they aren’t worth correcting.)
Powerful AI will be developed by large organizations (companies or governments), which tend to be very risk averse and so will ensure safety automatically.
Timelines are long and we can’t do much useful work on the problem today.
There are probably more I find compelling, I did not spend much time on this.
Some of what seems to me to be good arguments against entering the field, depending on what you include as the field.
We may live in a world where AI safety is either easy, or almost impossible to solve. In such cases it may be better to work e.g. on global coordination or rationality of leaders
It may be the case the “near-term” issues with AI will transform the world in a profound way / are big enough to pose catastrophic risks, and given the shorter timelines, and better tractability, they are higher priority. (For example, you can imagine technological unemployment + addictive narrow AI aided VR environments + decay of shared epistemology leading to unraveling of society. Or narrow AI accelerating biorisk.)
It may be the case the useful work on reduction of AI risk requires very special talent / judgment calibrated in special ways / etc. and the many people who want to enter the field will mostly harm the field, because the people who should start working on it will be drowned out by the noise created by the large mass.
(Note: I do not endorse the arguments. Also they are not answering the part about worrying.)
Scott Aaronson wrote about it last year, and it might count as an answer, especially the last paragraph:
Wow, that’s a very shallow and short-sighted reason for wanting AI to come sooner, especially since this is a global issue and not just American. I guess there’s a difference between intelligence and wisdom.
A few others have pointed to this, but I’d say more fully that to me the main reason not to worry about AI risk is that AI alignment is likely intractable in theory (although in practice I think we can dramatically reduce the space of AI minds we might create by removing lots of possibilities that are obviously unaligned and thus increase the probability we end up with an AI that’s friendly to humanity anyway). This problem is that we’re asking an AI to reason correctly about something we don’t have a good way to observe and measure (human values), and those methods we do have produce unwanted results via Goodharting. Of course this is, as I think of it, a reason not to worry, but it is not a reason not to work on the problem, since even if we will inevitable live with non-trivial risk of AI-induced extinction, we can still reduce that risk.
Not exactly what you were meaning to ask for maybe but I think contributes some flavor to the idea we can still work on a problem and care about it even if we don’t much worry about it.