I’ve always disliked the idea of people who assess risk differently being called “evil.” There’s a big difference between “I want to kill everyone, so I’m creating technology X” and “I disagree with the risk assessment of this technology and want to make the world a better place, so I’m creating it.”
I think the lesswrong community has become too radicalized in considering itself so epistemologically right that it calls people evil who simply disagree with them about the probability of p(doom).
UPD: The problem with this radicalization of the community is also that it repels many AI researchers who do not consider themselves evil and are less inclined to change their point of view on the risks posed by people who consider them evil and are probably not averse to killing them for disagreeing with them.
I actually think there are more people like that on LessWrong; the disagreement score on both their comment and this post has been going up and down a lot, so I think there is high variance. That is, I think there is a genuinely divisive conflict on LessWrong as to whether radicalization to the degree of calling AI developers “murderers” is good thing. (My position: it’s clearly a hyperbolic and thus irresponsible and/or bad-faith way to use the term ‘murder’.)
I did not call anyone a murderer in this thread. I did ask about it a month ago but replies convinced me it wasn’t appropriate. Although I see how even using it as an analogy could cause confusion. I’ll edit my comment.
I personally strong-downvoted the parent comment. When I remove my strong-downvote from it, I discover that the combined rest of LessWrong voted that comment to a total of +18 karma and +2 agreement as of me writing this.
I do in fact consider it somewhat alarming that the combined rest of LessWrong on net agrees with this comment and wants to reward the author for posting it.
Why are you asking me to put my own desire not to be killed along with everyone I care about at the bottom of my priorities list, below protecting the feelings of AI researchers? That is an insane request and I’m obviously not willing to do it.
I’d answer differently to Forza: I wouldn’t ask you to move it on your priorities list, I’d ask you to recognize that properly understood, prioritizing it means you do care about the feelings of AI researchers, and that you are making a mistake to treat their behavior as opaque. I’d like you to be trying to get them to stop more competently, which I don’t think involves telling them to feel like murderers separately from convincing them of the problem, because humans have known mental immune responses to being told how to feel in ways that are not justified by evidence they can directly process. I don’t mean to request a general update about all of your behavior, but I think your comments since my last ones don’t show evidence of having understood why I replied to you the way I did, or why I believe that the common memetic anger-pattern you are exhibiting here is a dominated strategy.
also, I explicitly do not claim that there is no memetic response warranted, nor that you can’t be mad. I just want you to recognize that people who might, in the full accounting of things, in fact qualify as ending up becoming the cause of mass death down the line, are in fact currently justified in being unsure whether that’s the case, and so being verbally confident at them is not a move that would be expected to change their behavior. Your pattern seems like one whose nearest effective variant is mass-movement-building, and I do think there are forms of that which can be effective; I think those will be the most effective if they’re highly palatable to contributors at labs while also not sacrificing moral clarity or factual-justification-to-an-uninformed-mind. I think you’re currently trying only for moral clarity, and moral clarity that results in autoimmune responses is in my view an unambiguously dominated strategy.
I’m not talking to AI researchers. I don’t think there’s any world where every AI researcher can talked into quitting, there will always be people lining up around the block to screw over humanity for 8+-figure amounts of money. This is why I think they have to be stopped. Talking an individual researcher into quitting may be good for that individual’s conscience, but only lengthens timelines by the difference between that person’s competence and their replacement, which probably isn’t enough to be worth much.
I don’t see why movement-building would be more effective if it’s highly palatable to contributors at labs, it seems more like the opposite is true, but feel free to explain.
because you might be wrong, or people might genuinely not understand that you’re right? There are many people who believe that their lives are in danger of a lot of things: vaccines against covid, GMOs, climate change, etc. They also sincerely believe that they are right and can consider scientists evil because they endanger their lives and the lives of their family. But the problem is that if they start equating any people who disagree with them with the evil that deliberately wants to destroy them, it radicalizes them and promoting enmity. Is this a rational approach when every person, confident that technology N can kill him, starts openly hating the scientists who develop it?
This is drawing a false equivalence between AI risk and random crackpot beliefs, which is dishonest. You won’t find accomplished scientists saying vaccines against covid, GMOs, or climate change have a substantial likelihood of killing everyone. Also it’s not relevant whether someone who’s going to kill me is evil and I’m not talking about that, I want them to be stopped from killing me whether they’re evil or not, and whether they can be talked out of it or not.
I’ve always disliked the idea of people who assess risk differently being called “evil.” There’s a big difference between “I want to kill everyone, so I’m creating technology X” and “I disagree with the risk assessment of this technology and want to make the world a better place, so I’m creating it.”
I think the lesswrong community has become too radicalized in considering itself so epistemologically right that it calls people evil who simply disagree with them about the probability of p(doom).
UPD: The problem with this radicalization of the community is also that it repels many AI researchers who do not consider themselves evil and are less inclined to change their point of view on the risks posed by people who consider them evil and are probably not averse to killing them for disagreeing with them.
You are responding to one person who received heavy disagreement in comments.
I actually think there are more people like that on LessWrong; the disagreement score on both their comment and this post has been going up and down a lot, so I think there is high variance. That is, I think there is a genuinely divisive conflict on LessWrong as to whether radicalization to the degree of calling AI developers “murderers” is good thing. (My position: it’s clearly a hyperbolic and thus irresponsible and/or bad-faith way to use the term ‘murder’.)
I did not call anyone a murderer in this thread. I did ask about it a month ago but replies convinced me it wasn’t appropriate. Although I see how even using it as an analogy could cause confusion. I’ll edit my comment.
I personally strong-downvoted the parent comment. When I remove my strong-downvote from it, I discover that the combined rest of LessWrong voted that comment to a total of +18 karma and +2 agreement as of me writing this.
I do in fact consider it somewhat alarming that the combined rest of LessWrong on net agrees with this comment and wants to reward the author for posting it.
Why are you asking me to put my own desire not to be killed along with everyone I care about at the bottom of my priorities list, below protecting the feelings of AI researchers? That is an insane request and I’m obviously not willing to do it.
I’d answer differently to Forza: I wouldn’t ask you to move it on your priorities list, I’d ask you to recognize that properly understood, prioritizing it means you do care about the feelings of AI researchers, and that you are making a mistake to treat their behavior as opaque. I’d like you to be trying to get them to stop more competently, which I don’t think involves telling them to feel like murderers separately from convincing them of the problem, because humans have known mental immune responses to being told how to feel in ways that are not justified by evidence they can directly process. I don’t mean to request a general update about all of your behavior, but I think your comments since my last ones don’t show evidence of having understood why I replied to you the way I did, or why I believe that the common memetic anger-pattern you are exhibiting here is a dominated strategy.
also, I explicitly do not claim that there is no memetic response warranted, nor that you can’t be mad. I just want you to recognize that people who might, in the full accounting of things, in fact qualify as ending up becoming the cause of mass death down the line, are in fact currently justified in being unsure whether that’s the case, and so being verbally confident at them is not a move that would be expected to change their behavior. Your pattern seems like one whose nearest effective variant is mass-movement-building, and I do think there are forms of that which can be effective; I think those will be the most effective if they’re highly palatable to contributors at labs while also not sacrificing moral clarity or factual-justification-to-an-uninformed-mind. I think you’re currently trying only for moral clarity, and moral clarity that results in autoimmune responses is in my view an unambiguously dominated strategy.
see also btfc’s comments on soft language
I’m not talking to AI researchers. I don’t think there’s any world where every AI researcher can talked into quitting, there will always be people lining up around the block to screw over humanity for 8+-figure amounts of money. This is why I think they have to be stopped. Talking an individual researcher into quitting may be good for that individual’s conscience, but only lengthens timelines by the difference between that person’s competence and their replacement, which probably isn’t enough to be worth much.
I don’t see why movement-building would be more effective if it’s highly palatable to contributors at labs, it seems more like the opposite is true, but feel free to explain.
because you might be wrong, or people might genuinely not understand that you’re right? There are many people who believe that their lives are in danger of a lot of things: vaccines against covid, GMOs, climate change, etc. They also sincerely believe that they are right and can consider scientists evil because they endanger their lives and the lives of their family. But the problem is that if they start equating any people who disagree with them with the evil that deliberately wants to destroy them, it radicalizes them and promoting enmity. Is this a rational approach when every person, confident that technology N can kill him, starts openly hating the scientists who develop it?
This is drawing a false equivalence between AI risk and random crackpot beliefs, which is dishonest. You won’t find accomplished scientists saying vaccines against covid, GMOs, or climate change have a substantial likelihood of killing everyone. Also it’s not relevant whether someone who’s going to kill me is evil and I’m not talking about that, I want them to be stopped from killing me whether they’re evil or not, and whether they can be talked out of it or not.