Bayesian Judo

You can have some fun with people whose an­ti­cip­a­tions get out of sync with what they be­lieve they be­lieve.

I was once at a din­ner party, try­ing to ex­plain to a man what I did for a liv­ing, when he said: “I don’t be­lieve Ar­ti­fi­cial In­tel­li­gence is pos­sible be­cause only God can make a soul.”

At this point I must have been di­vinely in­spired, be­cause I in­stantly re­spon­ded: “You mean if I can make an Ar­ti­fi­cial In­tel­li­gence, it proves your re­li­gion is false?”

He said, “What?”

I said, “Well, if your re­li­gion pre­dicts that I can’t pos­sibly make an Ar­ti­fi­cial In­tel­li­gence, then, if I make an Ar­ti­fi­cial In­tel­li­gence, it means your re­li­gion is false. Either your re­li­gion al­lows that it might be pos­sible for me to build an AI; or, if I build an AI, that dis­proves your re­li­gion.”

There was a pause, as the one real­ized he had just made his hy­po­thesis vul­ner­able to falsi­fic­a­tion, and then he said, “Well, I didn’t mean that you couldn’t make an in­tel­li­gence, just that it couldn’t be emo­tional in the same way we are.”

I said, “So if I make an Ar­ti­fi­cial In­tel­li­gence that, without be­ing de­lib­er­ately pre­pro­grammed with any sort of script, starts talk­ing about an emo­tional life that sounds like ours, that means your re­li­gion is wrong.”

He said, “Well, um, I guess we may have to agree to dis­agree on this.”

I said: “No, we can’t, ac­tu­ally. There’s a the­orem of ra­tion­al­ity called Au­mann’s Agree­ment The­orem which shows that no two ra­tion­al­ists can agree to dis­agree. If two people dis­agree with each other, at least one of them must be do­ing some­thing wrong.”

We went back and forth on this briefly. Fin­ally, he said, “Well, I guess I was really try­ing to say that I don’t think you can make some­thing eternal.”

I said, “Well, I don’t think so either! I’m glad we were able to reach agree­ment on this, as Au­mann’s Agree­ment The­orem re­quires.” I stretched out my hand, and he shook it, and then he wandered away.

A wo­man who had stood nearby, listen­ing to the con­ver­sa­tion, said to me gravely, “That was beau­ti­ful.”

“Thank you very much,” I said.

Part of the se­quence Mys­ter­i­ous An­swers to Mys­ter­i­ous Questions

Next post: “Pro­fess­ing and Cheer­ing

Pre­vi­ous post: “Be­lief in Be­lief