I’ve always consider the psychological critiques of AI risk (eg “the singularity is just rapture of the nerds”) to be very weak ad hominems. However, they might be relevant for parts of the AI risk thesis that depend on the judgements of the people presenting it. The most relevant part would be in checking whether people have fully considered the arguments against their position, and gone out to find more such arguments.
The argument is that people who talk about the singularity in general or AI risk (the hard-takeoff FOOM scenario) are privileging some low-probability hypotheses based on intuitions that come either directly from religion or from some underlying psychological mechanisms that also generate religious beliefs.
Most beliefs of this kind are wrong. They tend to be unparsimonious. Hence, when presented with a claim of this kind, before we look at the evidence or specific arguments, we should infer at first that the claim is likely wrong. Strong evidence or strong arguments would “screen off” this effect, while lack of evidence or weak arguments based on subjective estimates would not.
I’ve always consider the psychological critiques of AI risk (eg “the singularity is just rapture of the nerds”) to be very weak ad hominems. However, they might be relevant for parts of the AI risk thesis that depend on the judgements of the people presenting it. The most relevant part would be in checking whether people have fully considered the arguments against their position, and gone out to find more such arguments.
Autist people like to put one thing on the other (see more about autism and repetive behaviour in https://www.autismspeaks.org/science/science-news/study-suggests-repetitive-behaviors-emerge-early-autism) Recursive self-improvement is the same idea on higher level. That is why nerds (me too) may overestimate the probability of such type of AI. Sorry to said that.
The argument is that people who talk about the singularity in general or AI risk (the hard-takeoff FOOM scenario) are privileging some low-probability hypotheses based on intuitions that come either directly from religion or from some underlying psychological mechanisms that also generate religious beliefs.
Most beliefs of this kind are wrong. They tend to be unparsimonious. Hence, when presented with a claim of this kind, before we look at the evidence or specific arguments, we should infer at first that the claim is likely wrong. Strong evidence or strong arguments would “screen off” this effect, while lack of evidence or weak arguments based on subjective estimates would not.