Bayesian Judo

You can have some fun with peo­ple whose an­ti­ci­pa­tions get out of sync with what they be­lieve they be­lieve.

I was once at a din­ner party, try­ing to ex­plain to a man what I did for a liv­ing, when he said: “I don’t be­lieve Ar­tifi­cial In­tel­li­gence is pos­si­ble be­cause only God can make a soul.”

At this point I must have been di­v­inely in­spired, be­cause I in­stantly re­sponded: “You mean if I can make an Ar­tifi­cial In­tel­li­gence, it proves your re­li­gion is false?”

He said, “What?”

I said, “Well, if your re­li­gion pre­dicts that I can’t pos­si­bly make an Ar­tifi­cial In­tel­li­gence, then, if I make an Ar­tifi­cial In­tel­li­gence, it means your re­li­gion is false. Either your re­li­gion al­lows that it might be pos­si­ble for me to build an AI; or, if I build an AI, that dis­proves your re­li­gion.”

There was a pause, as the one re­al­ized he had just made his hy­poth­e­sis vuln­er­a­ble to falsifi­ca­tion, and then he said, “Well, I didn’t mean that you couldn’t make an in­tel­li­gence, just that it couldn’t be emo­tional in the same way we are.”

I said, “So if I make an Ar­tifi­cial In­tel­li­gence that, with­out be­ing de­liber­ately pre­pro­grammed with any sort of script, starts talk­ing about an emo­tional life that sounds like ours, that means your re­li­gion is wrong.”

He said, “Well, um, I guess we may have to agree to dis­agree on this.”

I said: “No, we can’t, ac­tu­ally. There’s a the­o­rem of ra­tio­nal­ity called Au­mann’s Agree­ment The­o­rem which shows that no two ra­tio­nal­ists can agree to dis­agree. If two peo­ple dis­agree with each other, at least one of them must be do­ing some­thing wrong.”

We went back and forth on this briefly. Fi­nally, he said, “Well, I guess I was re­ally try­ing to say that I don’t think you can make some­thing eter­nal.”

I said, “Well, I don’t think so ei­ther! I’m glad we were able to reach agree­ment on this, as Au­mann’s Agree­ment The­o­rem re­quires.” I stretched out my hand, and he shook it, and then he wan­dered away.

A woman who had stood nearby, listen­ing to the con­ver­sa­tion, said to me gravely, “That was beau­tiful.”

“Thank you very much,” I said.

Part of the se­quence Mys­te­ri­ous An­swers to Mys­te­ri­ous Questions

Next post: “Pro­fess­ing and Cheer­ing

Pre­vi­ous post: “Belief in Belief