Make sure you back up your comment, if you value it.
The mild LW censor is more subtle than that. Comments can continue to exist but do not show up unless you find the right path to them.
It’s apparent that to have a sane policy on this matter, Eliezer would have to change his mind. I cannot tell whether the existing policy is mainly supposed to prevent people from thinking scary thoughts, for the sake of their own well-being, or whether there is some genuine fear that possible AIs in the future will malevolently affect the past by being sketchily imagined in the present—which is absurd. Or maybe it’s some other variation on this idea which we’re all supposed to be tiptoeing around. But the effect of the censorship (however mild it is) is to make people unable to think and talk about the problem in a rational and uninhibited manner.
I really think that the key issue is the possibility of transhuman torture, and whether we permit that to even be mentioned. The current policy seems to be, that I can talk about the possibility of a maximally unfriendly postsingularity AI torturing the human race for millions of years, but I am not allowed to talk about whether a proposed information channel, whereby a possible but not yet existent AI supposedly threatens people with this in the present, makes any sense at all, because just thinking about it is traumatic for some people. I submit that this policy is inconsistent. The proposed information channel does not actually make sense, and in any case all the trauma is contained in the raw possibility of transhuman torture occurring to us, some day in the future. You shouldn’t need the extra icing of quasi-paranormal influences to find that possibility scary.
We should separate these two factors—the mechanics of the information channel, and the terror of transhuman torture—and decide separately (1) whether the proposed mechanism makes sense (2) whether the topic of transhuman torture, in any form, is just too psychologically dangerous to be publicly discussed. I say No to both of these.
As I understand the original posting and Eliezer’s response to it, the problem is not that some over-delicate souls might be distressed at a hypothetical danger. The (alleged) real problem is far worse: it is that thinking about these scenarios is the very thing that makes you vulnerable to them. And to twist the knife further, the problem isn’t limited to UFAIs. You might end up being tortured by a FAI, if you didn’t manage to think about these things in just the right way. Better to remain safely ignorant—if you can, having read just this much.
I can’t resist pointing out a religious analogue. There is a Christian belief that people who lived and died without the opportunity to hear the Word of God may still be saved if they nevertheless lived good lives in ignorance of the divine commandment. (Historically, I think the purpose of this doctrine was to protect the writings of the ancient Greeks and Romans from wholesale condemnation and destruction, but that’s by the way.) However, people who have had the opportunity to hear the Good News but reject it are damned without mercy. In God’s eyes they are worse than the most depraved of those who were ignorant through no fault of their own.
Surely that depends on exactly what you define “friendly” to mean.
It certainly seems to. Somewhere on my list of “ways to stop an AI from torturing me for 10 million years” is “find anyone who is in the process of creating an AI that will torture me and kill them”. I’m not overly concerned what name they give it.
Since Eliezer considers it rational to prefer TORTURE to SPECKS, an FAI to his specification would presumably do the same. In either case, too bad if you’re the one who gets TORTUREd. Maybe 3^^^3 people to SPECK will never be created, but what is one person compared with even the mere bazillions that FAI-assisted humanity might produce in mere billions of years? You need to make very sure you’re one of the elect before creating God.
The mild LW censor is more subtle than that. Comments can continue to exist but do not show up unless you find the right path to them.
It’s apparent that to have a sane policy on this matter, Eliezer would have to change his mind. I cannot tell whether the existing policy is mainly supposed to prevent people from thinking scary thoughts, for the sake of their own well-being, or whether there is some genuine fear that possible AIs in the future will malevolently affect the past by being sketchily imagined in the present—which is absurd. Or maybe it’s some other variation on this idea which we’re all supposed to be tiptoeing around. But the effect of the censorship (however mild it is) is to make people unable to think and talk about the problem in a rational and uninhibited manner.
I really think that the key issue is the possibility of transhuman torture, and whether we permit that to even be mentioned. The current policy seems to be, that I can talk about the possibility of a maximally unfriendly postsingularity AI torturing the human race for millions of years, but I am not allowed to talk about whether a proposed information channel, whereby a possible but not yet existent AI supposedly threatens people with this in the present, makes any sense at all, because just thinking about it is traumatic for some people. I submit that this policy is inconsistent. The proposed information channel does not actually make sense, and in any case all the trauma is contained in the raw possibility of transhuman torture occurring to us, some day in the future. You shouldn’t need the extra icing of quasi-paranormal influences to find that possibility scary.
We should separate these two factors—the mechanics of the information channel, and the terror of transhuman torture—and decide separately (1) whether the proposed mechanism makes sense (2) whether the topic of transhuman torture, in any form, is just too psychologically dangerous to be publicly discussed. I say No to both of these.
As I understand the original posting and Eliezer’s response to it, the problem is not that some over-delicate souls might be distressed at a hypothetical danger. The (alleged) real problem is far worse: it is that thinking about these scenarios is the very thing that makes you vulnerable to them. And to twist the knife further, the problem isn’t limited to UFAIs. You might end up being tortured by a FAI, if you didn’t manage to think about these things in just the right way. Better to remain safely ignorant—if you can, having read just this much.
I can’t resist pointing out a religious analogue. There is a Christian belief that people who lived and died without the opportunity to hear the Word of God may still be saved if they nevertheless lived good lives in ignorance of the divine commandment. (Historically, I think the purpose of this doctrine was to protect the writings of the ancient Greeks and Romans from wholesale condemnation and destruction, but that’s by the way.) However, people who have had the opportunity to hear the Good News but reject it are damned without mercy. In God’s eyes they are worse than the most depraved of those who were ignorant through no fault of their own.
Some “Good News”, and some “Friendliness”!
Surely that depends on exactly what you define “friendly” to mean.
It certainly seems to. Somewhere on my list of “ways to stop an AI from torturing me for 10 million years” is “find anyone who is in the process of creating an AI that will torture me and kill them”. I’m not overly concerned what name they give it.
Since Eliezer considers it rational to prefer TORTURE to SPECKS, an FAI to his specification would presumably do the same. In either case, too bad if you’re the one who gets TORTUREd. Maybe 3^^^3 people to SPECK will never be created, but what is one person compared with even the mere bazillions that FAI-assisted humanity might produce in mere billions of years? You need to make very sure you’re one of the elect before creating God.
The parallels with Christian theology just keep coming. Thanks to Timeless Decision Theory, you were either saved or damned from the beginning. When you attain to the correct dispositions to be immune to counterfactual blackmail, you do not become saved, but discover that you always were. And do not delay, for “Every day brings you nearer to everlasting torments or felicity.” “Your transgressions have sent up to heaven a cry for vengeance. You are actually under the curse of the Almighty.” The Bible makes a lot of sense read as a garbled account of an AI that played around with the human race for a while and then went away.
Which brings us back to… who is creating this unfriendly AI that is going to torture me and where do they live?
Probably the same people who push fat people under trolleys. I wonder what sort of AI Peter Singer would want to create?
So: I wonder if you got “mildly censored”. All I can see now is “comment deleted”.