‘This particular argument’ was meant to be unspecific since I was talking about an aspect of Ideological Turing Tests (or fights generally) that doesn’t hinge on what you’re fighting about.
If you think your interlocutor is obviously wrong, and there’s nothing for you to learn by trying to model him more accurately, you may be wrong about that! The flaw in his thinking that’s causing him to ignore data is probably native to you as well. Putting in the work to spot it and to observe what defensive strategies he’s using to avoid spotting it may cause a queasy feeling of recognition that you used the same kinds of language/flinches/etc in a different recent argument, and now you should go back and check your data.
I think the flaw is that humans copy from the culture they immerse themselves in. Do you think you would have come to convert to Catholicism without engaging with Catholicism over a large time span? Assuming this, shouldn’t I avoid confronting/studying any religion in depth, that I’m not already immunized to? How is a rationalist to act when every intense engagement with a belief, true or not, makes it more likely they’ll adopt that belief?
PS: unrelatedly, would you let a FAI convince you that Catholicism is false? I think I’d let a FAI convince me that Catholicism is true, assuming it was built by an uninterested third party (neither the Catholic church nor, say, Dawkins). Should MIRI hire religious people to assure people that their FAI was not built unduly biased towards atheism?
Of course, but people may not believe out of hand that a superintelligence built by atheists saying atheism is correct is not just parroting its creators. Might be important in the take-off phase to have that extra bit of public trust.
Once you have FAI, you’re set. There is nothing left you need to do. If something needs to be done, the FAI will know better than who what has to be done and how to do it. If it turns out it should have been written by Christians, it will tell some Christians how to write an FAI and make sure they do it correctly. Worrying about what to do after* running the program is like taking a cup of water with you as you flee your burning house, so that when the fire department arrive you can help out.
*except those things the FAI judges it would be good for you to need to do, which are not relevant here.
‘This particular argument’ was meant to be unspecific since I was talking about an aspect of Ideological Turing Tests (or fights generally) that doesn’t hinge on what you’re fighting about.
If you think your interlocutor is obviously wrong, and there’s nothing for you to learn by trying to model him more accurately, you may be wrong about that! The flaw in his thinking that’s causing him to ignore data is probably native to you as well. Putting in the work to spot it and to observe what defensive strategies he’s using to avoid spotting it may cause a queasy feeling of recognition that you used the same kinds of language/flinches/etc in a different recent argument, and now you should go back and check your data.
I think the flaw is that humans copy from the culture they immerse themselves in. Do you think you would have come to convert to Catholicism without engaging with Catholicism over a large time span? Assuming this, shouldn’t I avoid confronting/studying any religion in depth, that I’m not already immunized to? How is a rationalist to act when every intense engagement with a belief, true or not, makes it more likely they’ll adopt that belief?
PS: unrelatedly, would you let a FAI convince you that Catholicism is false? I think I’d let a FAI convince me that Catholicism is true, assuming it was built by an uninterested third party (neither the Catholic church nor, say, Dawkins). Should MIRI hire religious people to assure people that their FAI was not built unduly biased towards atheism?
If it’s ‘unduly’ anything it’s not FAI.
Of course, but people may not believe out of hand that a superintelligence built by atheists saying atheism is correct is not just parroting its creators. Might be important in the take-off phase to have that extra bit of public trust.
Once you have FAI, you’re set. There is nothing left you need to do. If something needs to be done, the FAI will know better than who what has to be done and how to do it. If it turns out it should have been written by Christians, it will tell some Christians how to write an FAI and make sure they do it correctly. Worrying about what to do after* running the program is like taking a cup of water with you as you flee your burning house, so that when the fire department arrive you can help out.
*except those things the FAI judges it would be good for you to need to do, which are not relevant here.