Some of those people have been researching AI for decades, wrote hundreds of papers that have been cited many thousands of times.
If people are updating correctly they should have already factored in to their beliefs the fact that the SIAI position isn’t anywhere near the consensus in university artificial intelligence research. Given the assumption that these people are highly intelligent and well versed in their fields one should probably disbelieve the SIAI position at the outset because one expects that upon hearing from a “mainstream” AI researcher one would learn the reasons why SIAI is wrong. But if you then read a conversation between a “mainstream” AI researcher and an SIAI researcher and the former can’t explain why the latter is wrong then you better start updating.
They have probably thought about everything you know long before you and dismissed it.
I’m sure this is true when it comes to, say, programming a particular sort of artificial intelligence. But you are vastly overestimating how much thought scientists and engineers put into broad, philosophical concerns involving their fields. With few exceptions mathematicians don’t spend time thinking about the reality of infinite sets, physicists don’t spend time thinking about interpretations of quantum mechanics, computer programmers don’t spend time thinking about the Church-Turing-Deutsch principle etc.
But if you then read a conversation between a “mainstream” AI researcher and an SIAI researcher and the former can’t explain why the latter is wrong then you better start updating.
His arguments were not worse than Luke’s arguments if you ignore all the links, which he has no reason to read. He said that he does not believe that it is possible to restrict an AI the way that SI does imagine and still produce a general intelligence. He believes that the most promising route is AI that can learn by being taught.
In combination with his doubts about uncontrollable superintelligence, that position is not incoherent. You can also not claim, given this short dialogue, that he did not explain why SI is wrong.
But you are vastly overestimating how much thought scientists and engineers put into broad, philosophical concerns involving their fields.
That’s not what I was referring to. I doubt they have thought a lot about AI risks. What I meant is that they have likely thought about the possibility of recursive self-improvement and uncontrollable superhuman intelligence.
If an AI researcher tells you that he believes that AI risks are not a serious issue because they do not believe that AI can get out of control for technical reasons and you reply that they have not thought about AI drives and the philosophical reasons for why superhuman AI will pose a risk, then you created a straw man. Which is the usual tactic employed here.
If people are updating correctly they should have already factored in to their beliefs the fact that the SIAI position isn’t anywhere near the consensus in university artificial intelligence research. Given the assumption that these people are highly intelligent and well versed in their fields one should probably disbelieve the SIAI position at the outset because one expects that upon hearing from a “mainstream” AI researcher one would learn the reasons why SIAI is wrong. But if you then read a conversation between a “mainstream” AI researcher and an SIAI researcher and the former can’t explain why the latter is wrong then you better start updating.
I’m sure this is true when it comes to, say, programming a particular sort of artificial intelligence. But you are vastly overestimating how much thought scientists and engineers put into broad, philosophical concerns involving their fields. With few exceptions mathematicians don’t spend time thinking about the reality of infinite sets, physicists don’t spend time thinking about interpretations of quantum mechanics, computer programmers don’t spend time thinking about the Church-Turing-Deutsch principle etc.
His arguments were not worse than Luke’s arguments if you ignore all the links, which he has no reason to read. He said that he does not believe that it is possible to restrict an AI the way that SI does imagine and still produce a general intelligence. He believes that the most promising route is AI that can learn by being taught.
In combination with his doubts about uncontrollable superintelligence, that position is not incoherent. You can also not claim, given this short dialogue, that he did not explain why SI is wrong.
That’s not what I was referring to. I doubt they have thought a lot about AI risks. What I meant is that they have likely thought about the possibility of recursive self-improvement and uncontrollable superhuman intelligence.
If an AI researcher tells you that he believes that AI risks are not a serious issue because they do not believe that AI can get out of control for technical reasons and you reply that they have not thought about AI drives and the philosophical reasons for why superhuman AI will pose a risk, then you created a straw man. Which is the usual tactic employed here.