What are the most egregious examples of SI’s arrogance?
Public tantrums, shouting and verbal abuse. Those are status displays that pay off for tribal chieftans and some styles of gang leader. They aren’t appropriate for leaders of intellectually oriented charities. Eliezer thinking he can get away with that is the biggest indicator of arrogance that I’ve noticed thus far.
To be fair, while I personally do perceive the SIAI as being arrogant, I haven’t seen any public tantrums. As far as I can tell, all their public discourse has been quite civil.
To be fair, while I personally do perceive the SIAI as being arrogant, I haven’t seen any public tantrums. As far as I can tell, all their public discourse has been quite civil.
The most significant example was the Roko incident. The relevant threads and comments were all censored during the later part of his tantrum. Not a good day in the life of Eliezer’s reputation.
Fair enough; I was unaware of the Roko incident (understandably so, since apparently it was Sovieted from history). I have now looked it up elsewhere, though. Thanks for the info.
I tried to look up this Roko incident, and from what I could see, Eliezer just acted crazily towards saying something Eliezer thought was dangerous. So him deleting everything could be justified without him necessarily being egotistical.
I’m sort of pleased to see that I guessed roughly what this episode was about despite having arrived at LessWrong well after it unhappened.+ But if the Rationalwiki description is accurate, I’m now really confused about something new.
I was under the impression that Lesswrong was fairly big on the Litany of Gendlin. But an AI that could do the things Roko proposed (something I place vanishingly small probability, fortunately) could also retrospectively figure out who was being willfully ignorant or failing to reach rational conclusions for which they had sufficient priors.
It’s disconcerting, after watching so much criticism of the rest of humanity finding ways to rationalize around the “inevitability” of death, to see transhumanists finding ways to hide their minds from their own “inevitable” conclusions.
+Since most people who would care about this subject at all have probably read Three Worlds Collide, I think this episode should be referred to as The Confessar Vanishes, but my humor may be idiosyncratic even for this crowd.
The primary issue with the Roko matter wasn’t as much that an AI might actually do but that the relevant memes could cause some degree of stress in neurotic individuals. At the time when it occurred there were at least two people in the general SI/LW cluster who were apparently deeply disturbed by the thought. I expect that the sort who would be vulnerable would be the same sort who if they were religious would lose sleep over the possibility of going to hell.
The primary issue with the Roko matter wasn’t as much that an AI might actually do but that the relevant memes could cause some degree of stress in neurotic individuals.
The original reasons given:
Meanwhile I’m banning this post so that it doesn’t (a) give people horrible nightmares and (b) give distant superintelligences a motive to follow through on blackmail against people dumb enough to think about them in sufficient detail, though, thankfully, I doubt anyone dumb enough to do this knows the sufficient detail. (I’m not sure I know the sufficient detail.)
...and further:
For those who have no idea why I’m using capital letters for something that just sounds like a random crazy idea, and worry that it means I’m as crazy as Roko, the gist of it was that he just did something that potentially gives superintelligences an increased motive to do extremely evil things in an attempt to blackmail us. It is the sort of thing you want to be EXTREMELY CONSERVATIVE about NOT DOING.
I should have known that Rationalwiki would be the place to look for dirt on Eliezer. Thanks for the link.
Wow, that was fascinating reading. I still don’t think that we could call it a tantrum of Eliezer’s. I mean, I have no doubt he acted like a dick, but he probably at least thought that the Roko guy was being stupid.
I still don’t think that we could call it a tantrum of Eliezer’s.
Whatever you choose to call it the act of shouting at people and calling them names is the kind of thing that looks bad to me. I think Eliezer would look better if he didn’t shout or call people names.
but he probably at least thought that the Roko guy was being stupid.
Of course he did. Lack of sincerity is not the problem here. The belief that the other person is stupid and, more importantly, the belief that if he thinks other people are being stupid it is right and appropriate for him to launch into an abusive hysterical tirade is the arrogance problem in this case.
I think Eliezer would look better if he didn’t shout or call people names.
I agree. Eliezer is occasionally a jerk, and it looks like this was one of those times. Also, I have no idea what went on and you do, so any disagreement from me is pretty dubious.
Nitpicking: I don’t think that’s how we should use the work tantrum. Tantrum makes it sound like someone criticized Eliezer and he got mad at them. (I suppose that might have happened, though...) I dunno. I just dislike your choice of words. I would have phrased it as “Eliezer should put more effort into not occasionally being an arrogant dick.”
The word ‘Tantrum’ invokes in my mind a picture of either a child or someone with an overwhelmingly high perception of their status responding to things not going their way by acting out emotionally in violation of usual norms of behavior that apply to everyone else.
I would have phrased it as “Eliezer should put more effort into not occasionally being an arrogant dick.”
I did not want to make that point. Acting out when things don’t go his way is a distinctly different behavior pattern with different connotations with respect to arrogance. I’m going to stick with tantrum because it just seems to be exactly what I’m trying to convey.
I think he did the right thing there. He did it badly and clumsily, but had I been in his place I’d have had a hard time getting a grip on my emotions, and we know how sensitive and emotional he is.
Rational Wiki are great guys. We try to watch our own step, but it’s nice to have someone else watching us too, who can understand and sympathize with what we do.
Public tantrums, shouting and verbal abuse. Those are status displays that pay off for tribal chieftans and some styles of gang leader. They aren’t appropriate for leaders of intellectually oriented charities. Eliezer thinking he can get away with that is the biggest indicator of arrogance that I’ve noticed thus far.
To be fair, while I personally do perceive the SIAI as being arrogant, I haven’t seen any public tantrums. As far as I can tell, all their public discourse has been quite civil.
The most significant example was the Roko incident. The relevant threads and comments were all censored during the later part of his tantrum. Not a good day in the life of Eliezer’s reputation.
Fair enough; I was unaware of the Roko incident (understandably so, since apparently it was Sovieted from history). I have now looked it up elsewhere, though. Thanks for the info.
I tried to look up this Roko incident, and from what I could see, Eliezer just acted crazily towards saying something Eliezer thought was dangerous. So him deleting everything could be justified without him necessarily being egotistical.
But can you elaborate on what happened, please?
Oh, of course. The deletion just explains Bug’s unfamiliarity, it isn’t an arrogance example itself.
Rationalwiki.
I’m sort of pleased to see that I guessed roughly what this episode was about despite having arrived at LessWrong well after it unhappened.+ But if the Rationalwiki description is accurate, I’m now really confused about something new.
I was under the impression that Lesswrong was fairly big on the Litany of Gendlin. But an AI that could do the things Roko proposed (something I place vanishingly small probability, fortunately) could also retrospectively figure out who was being willfully ignorant or failing to reach rational conclusions for which they had sufficient priors.
It’s disconcerting, after watching so much criticism of the rest of humanity finding ways to rationalize around the “inevitability” of death, to see transhumanists finding ways to hide their minds from their own “inevitable” conclusions.
+Since most people who would care about this subject at all have probably read Three Worlds Collide, I think this episode should be referred to as The Confessar Vanishes, but my humor may be idiosyncratic even for this crowd.
The primary issue with the Roko matter wasn’t as much that an AI might actually do but that the relevant memes could cause some degree of stress in neurotic individuals. At the time when it occurred there were at least two people in the general SI/LW cluster who were apparently deeply disturbed by the thought. I expect that the sort who would be vulnerable would be the same sort who if they were religious would lose sleep over the possibility of going to hell.
The original reasons given:
...and further:
(emphasis mine)
I should have known that Rationalwiki would be the place to look for dirt on Eliezer. Thanks for the link.
Wow, that was fascinating reading. I still don’t think that we could call it a tantrum of Eliezer’s. I mean, I have no doubt he acted like a dick, but he probably at least thought that the Roko guy was being stupid.
Whatever you choose to call it the act of shouting at people and calling them names is the kind of thing that looks bad to me. I think Eliezer would look better if he didn’t shout or call people names.
Of course he did. Lack of sincerity is not the problem here. The belief that the other person is stupid and, more importantly, the belief that if he thinks other people are being stupid it is right and appropriate for him to launch into an abusive hysterical tirade is the arrogance problem in this case.
I agree. Eliezer is occasionally a jerk, and it looks like this was one of those times. Also, I have no idea what went on and you do, so any disagreement from me is pretty dubious.
Nitpicking: I don’t think that’s how we should use the work tantrum. Tantrum makes it sound like someone criticized Eliezer and he got mad at them. (I suppose that might have happened, though...) I dunno. I just dislike your choice of words. I would have phrased it as “Eliezer should put more effort into not occasionally being an arrogant dick.”
The word ‘Tantrum’ invokes in my mind a picture of either a child or someone with an overwhelmingly high perception of their status responding to things not going their way by acting out emotionally in violation of usual norms of behavior that apply to everyone else.
I did not want to make that point. Acting out when things don’t go his way is a distinctly different behavior pattern with different connotations with respect to arrogance. I’m going to stick with tantrum because it just seems to be exactly what I’m trying to convey.
I think he did the right thing there. He did it badly and clumsily, but had I been in his place I’d have had a hard time getting a grip on my emotions, and we know how sensitive and emotional he is.
Rational Wiki are great guys. We try to watch our own step, but it’s nice to have someone else watching us too, who can understand and sympathize with what we do.