Most, if not all, human minds are vulnerable to hacking, eg by cults, religions, pseudoscience, etc. The minds of rationalists should be harder to hack than others.
Make a copy of a (would-be) rationalist, subject the copy to significant emotional stress, and then send missionaries his way.
The myths carried by the missionaries should be invented for the challenge so everyone can agree that they are false, but should, of course, be significantly more plausible than today’s religions.
“crack a rationalist” made me think of the AI-Box Experiment (“http://yudkowsky.net/singularity/aibox″) Maybe a rationality test could be something like how long the subject lasts as the gatekeeper before letting the AI out.
What ciphergoth said. Also, we can’t derive an ‘ought’ from an ‘is’ - we don’t actually know whether letting the AI out is the right thing to do (unless the contest had a stipulation that the AI was evil and the box keeper knew it, which I don’t remember being the case). Perhaps the rational thing is to let the AI out!
Further, this could also just be a test of stubbornness or patience. Which aren’t neither of them rationality. But good try anyway.
For the first objection, that the AI Box experiment has too many unknowns, let us instead construct an argument based on psychological tricks for any bad conclusion to try on the subject.
For the second objection, that this tests stubbornness rather than rationality, use a sequence of tests, some using tricks to argue for false conclusions, and some using Bayesian evidence for a good conclusion. The score should reward being convinced when, and only when, the subject should be convinced. Stubbornness can only meet half this requirement.
The task of compiling arguments of both types, which would not be readily available to the subject ahead of time, remains.
The means by which EY persuades people to let the AI out of the box are secret. We shouldn’t draw any conclusions from that experiment except that it is plausible to think a boxed AI could talk its way out of the box.
The same complaint applies to this comment as to the wife-cheating test. It may actually (under certain really bad circumstances) be rational (in the “winning” sense) to believe in religion.
I’ll be honest—my life has taken a sharp downturn since I deconverted. My theist girlfriend, with whom I was very much in love, couldn’t deal with this change in me, and after six months of painful vacillation, she left me for a co-worker. That was another six months ago, and I have been heartbroken, miserable, unfocused, and extremely ineffective since.
Perhaps this is an example of the valley of bad rationality of which PhilGoetz spoke, but I still hold my current situation higher in my preference ranking than happiness with false beliefs.
Thank you. You taught me (a large chunk of) everything I know, so that means a lot.
Honestly, thinking back, I suspect the best opportunity I ever had to deconvert her was when I myself did not yet identify as atheist—when the crisis of faith was still in full swing. I’d have been perceived as sharing my doubts, rather than as “attacking” her with arguments.
Of course, back then I feared atheism—I saw it as something terrible happening to me, that I should avoid doing to her. If I’d done a better job of leaving a line of retreat, I might have made better choices—I might have shared each doubt as it occurred to me, instead of winding up 30 inferential steps removed from the woman I loved.
(And no, explaining that there is an inferential distance between you greater than is likely to be encountered in the ancestral environment really does not help in a fight)
I’ve been thinking lately of trying to write something addressed specifically to those beginning to question their religions. Life doesn’t come with save points, but standing at the spot you went wrong, calling out advice to passers-by seems like the next best thing.
That last sentence is just ludicrously dense with both important advice and good tips for game design. It’s excellent, is what I’m saying, and thanks for writing it. :-)
My empathies: that happened to me about 6 years ago (though thankfully without as much visible vacillation).
My sister, who had some Cognitive Behaviour Therapy training, reminded me that relationships are forming and breaking all the time, and given I wasn’t unattractive and hadn’t retreated into monastic seclusion, it wasn’t rational to think I’d be alone for the rest of my life (she turned out to be right). That was helpful at the times when my feelings hadn’t completely got the better of me. I suppose we can be haunted by stuff that is real.
Thank you. I’ve been struggling with that haunting myself. I think part of the problem is that when you’re in a relationship long enough, you wind up with a term in your utility function for that person. And even if you know you could wind up with someone objectively better, better suited, the outcome doesn’t seem like good news to your mind. A job for self-modification, I suppose, even if it’s the slow, manual kind.
Here’s an immoral one: crack a rationalist
Most, if not all, human minds are vulnerable to hacking, eg by cults, religions, pseudoscience, etc. The minds of rationalists should be harder to hack than others.
Make a copy of a (would-be) rationalist, subject the copy to significant emotional stress, and then send missionaries his way.
The myths carried by the missionaries should be invented for the challenge so everyone can agree that they are false, but should, of course, be significantly more plausible than today’s religions.
Moral qualms aside, we should probably have a back-up plan just in case we don’t solve human uploading before we want to start testing.
“crack a rationalist” made me think of the AI-Box Experiment (“http://yudkowsky.net/singularity/aibox″) Maybe a rationality test could be something like how long the subject lasts as the gatekeeper before letting the AI out.
What ciphergoth said. Also, we can’t derive an ‘ought’ from an ‘is’ - we don’t actually know whether letting the AI out is the right thing to do (unless the contest had a stipulation that the AI was evil and the box keeper knew it, which I don’t remember being the case). Perhaps the rational thing is to let the AI out!
Further, this could also just be a test of stubbornness or patience. Which aren’t neither of them rationality. But good try anyway.
For the first objection, that the AI Box experiment has too many unknowns, let us instead construct an argument based on psychological tricks for any bad conclusion to try on the subject.
For the second objection, that this tests stubbornness rather than rationality, use a sequence of tests, some using tricks to argue for false conclusions, and some using Bayesian evidence for a good conclusion. The score should reward being convinced when, and only when, the subject should be convinced. Stubbornness can only meet half this requirement.
The task of compiling arguments of both types, which would not be readily available to the subject ahead of time, remains.
The means by which EY persuades people to let the AI out of the box are secret. We shouldn’t draw any conclusions from that experiment except that it is plausible to think a boxed AI could talk its way out of the box.
Brilliant.
The same complaint applies to this comment as to the wife-cheating test. It may actually (under certain really bad circumstances) be rational (in the “winning” sense) to believe in religion.
I’ll be honest—my life has taken a sharp downturn since I deconverted. My theist girlfriend, with whom I was very much in love, couldn’t deal with this change in me, and after six months of painful vacillation, she left me for a co-worker. That was another six months ago, and I have been heartbroken, miserable, unfocused, and extremely ineffective since.
Perhaps this is an example of the valley of bad rationality of which PhilGoetz spoke, but I still hold my current situation higher in my preference ranking than happiness with false beliefs.
You have my sympathy and my praise.
If anyone’s unusually good at deconversions, there might be a market for deconversion attempts aimed at the friends and family of atheists.
Thank you. You taught me (a large chunk of) everything I know, so that means a lot.
Honestly, thinking back, I suspect the best opportunity I ever had to deconvert her was when I myself did not yet identify as atheist—when the crisis of faith was still in full swing. I’d have been perceived as sharing my doubts, rather than as “attacking” her with arguments.
Of course, back then I feared atheism—I saw it as something terrible happening to me, that I should avoid doing to her. If I’d done a better job of leaving a line of retreat, I might have made better choices—I might have shared each doubt as it occurred to me, instead of winding up 30 inferential steps removed from the woman I loved.
(And no, explaining that there is an inferential distance between you greater than is likely to be encountered in the ancestral environment really does not help in a fight)
I’ve been thinking lately of trying to write something addressed specifically to those beginning to question their religions. Life doesn’t come with save points, but standing at the spot you went wrong, calling out advice to passers-by seems like the next best thing.
That last sentence is just ludicrously dense with both important advice and good tips for game design. It’s excellent, is what I’m saying, and thanks for writing it. :-)
Thanks XD
Please do.
Isn’t there already one to get people out of not widely accepted cults? The market might explode once public perception changes.
My empathies: that happened to me about 6 years ago (though thankfully without as much visible vacillation).
My sister, who had some Cognitive Behaviour Therapy training, reminded me that relationships are forming and breaking all the time, and given I wasn’t unattractive and hadn’t retreated into monastic seclusion, it wasn’t rational to think I’d be alone for the rest of my life (she turned out to be right). That was helpful at the times when my feelings hadn’t completely got the better of me. I suppose we can be haunted by stuff that is real.
Thank you. I’ve been struggling with that haunting myself. I think part of the problem is that when you’re in a relationship long enough, you wind up with a term in your utility function for that person. And even if you know you could wind up with someone objectively better, better suited, the outcome doesn’t seem like good news to your mind. A job for self-modification, I suppose, even if it’s the slow, manual kind.
Very glad to hear she was right =)