I think the flaw is that humans copy from the culture they immerse themselves in. Do you think you would have come to convert to Catholicism without engaging with Catholicism over a large time span? Assuming this, shouldn’t I avoid confronting/studying any religion in depth, that I’m not already immunized to? How is a rationalist to act when every intense engagement with a belief, true or not, makes it more likely they’ll adopt that belief?
PS: unrelatedly, would you let a FAI convince you that Catholicism is false? I think I’d let a FAI convince me that Catholicism is true, assuming it was built by an uninterested third party (neither the Catholic church nor, say, Dawkins). Should MIRI hire religious people to assure people that their FAI was not built unduly biased towards atheism?
Of course, but people may not believe out of hand that a superintelligence built by atheists saying atheism is correct is not just parroting its creators. Might be important in the take-off phase to have that extra bit of public trust.
Once you have FAI, you’re set. There is nothing left you need to do. If something needs to be done, the FAI will know better than who what has to be done and how to do it. If it turns out it should have been written by Christians, it will tell some Christians how to write an FAI and make sure they do it correctly. Worrying about what to do after* running the program is like taking a cup of water with you as you flee your burning house, so that when the fire department arrive you can help out.
*except those things the FAI judges it would be good for you to need to do, which are not relevant here.
I think the flaw is that humans copy from the culture they immerse themselves in. Do you think you would have come to convert to Catholicism without engaging with Catholicism over a large time span? Assuming this, shouldn’t I avoid confronting/studying any religion in depth, that I’m not already immunized to? How is a rationalist to act when every intense engagement with a belief, true or not, makes it more likely they’ll adopt that belief?
PS: unrelatedly, would you let a FAI convince you that Catholicism is false? I think I’d let a FAI convince me that Catholicism is true, assuming it was built by an uninterested third party (neither the Catholic church nor, say, Dawkins). Should MIRI hire religious people to assure people that their FAI was not built unduly biased towards atheism?
If it’s ‘unduly’ anything it’s not FAI.
Of course, but people may not believe out of hand that a superintelligence built by atheists saying atheism is correct is not just parroting its creators. Might be important in the take-off phase to have that extra bit of public trust.
Once you have FAI, you’re set. There is nothing left you need to do. If something needs to be done, the FAI will know better than who what has to be done and how to do it. If it turns out it should have been written by Christians, it will tell some Christians how to write an FAI and make sure they do it correctly. Worrying about what to do after* running the program is like taking a cup of water with you as you flee your burning house, so that when the fire department arrive you can help out.
*except those things the FAI judges it would be good for you to need to do, which are not relevant here.