The Strangest Thing An AI Could Tell You

Hu­man be­ings are all crazy. And if you tap on our brains just a lit­tle, we get so crazy that even other hu­mans no­tice. Anosog­nosics are one of my fa­vorite ex­am­ples of this; peo­ple with right-hemi­sphere dam­age whose left arms be­come par­a­lyzed, and who deny that their left arms are par­a­lyzed, com­ing up with ex­cuses when­ever they’re asked why they can’t move their arms.

A truly won­der­ful form of brain dam­age—it dis­ables your abil­ity to no­tice or ac­cept the brain dam­age. If you’re told out­right that your arm is par­a­lyzed, you’ll deny it. All the mar­velous ex­cuse-gen­er­at­ing ra­tio­nal­iza­tion fac­ul­ties of the brain will be mo­bi­lized to mask the dam­age from your own sight. As Yvain sum­ma­rized:

After a right-hemi­sphere stroke, she lost move­ment in her left arm but con­tin­u­ously de­nied it. When the doc­tor asked her to move her arm, and she ob­served it not mov­ing, she claimed that it wasn’t ac­tu­ally her arm, it was her daugh­ter’s. Why was her daugh­ter’s arm at­tached to her shoulder? The pa­tient claimed her daugh­ter had been there in the bed with her all week. Why was her wed­ding ring on her daugh­ter’s hand? The pa­tient said her daugh­ter had bor­rowed it. Where was the pa­tient’s arm? The pa­tient “turned her head and searched in a be­mused way over her left shoulder”.

I find it dis­turb­ing that the brain has such a sim­ple macro for ab­solute de­nial that it can be in­voked as a side effect of paral­y­sis. That a sin­gle whack on the brain can both dis­able a left-side mo­tor func­tion, and dis­able our abil­ity to rec­og­nize or ac­cept the dis­abil­ity. Other forms of brain dam­age also seem to both cause in­san­ity and dis­al­low recog­ni­tion of that in­san­ity—for ex­am­ple, when peo­ple in­sist that their friends have been re­placed by ex­act du­pli­cates af­ter dam­age to face-rec­og­niz­ing ar­eas.

And it re­ally makes you won­der...

...what if we all have some form of brain dam­age in com­mon, so that none of us no­tice some sim­ple and ob­vi­ous fact? As blatant, per­haps, as our left arms be­ing par­a­lyzed? Every time this fact in­trudes into our uni­verse, we come up with some ridicu­lous ex­cuse to dis­miss it—as ridicu­lous as “It’s my daugh­ter’s arm”—only there’s no sane doc­tor watch­ing to pur­sue the ar­gu­ment any fur­ther. (Would we all come up with the same ex­cuse?)

If the “ab­solute de­nial macro” is that sim­ple, and in­voked that eas­ily...

Now, sup­pose you built an AI. You wrote the source code your­self, and so far as you can tell by in­spect­ing the AI’s thought pro­cesses, it has no equiv­a­lent of the “ab­solute de­nial macro”—there’s no point dam­age that could in­flict on it the equiv­a­lent of anosog­nosia. It has re­dun­dant differ­ently-ar­chi­tected sys­tems, defend­ing in depth against cog­ni­tive er­rors. If one sys­tem makes a mis­take, two oth­ers will catch it. The AI has no func­tion­al­ity at all for de­liber­ate ra­tio­nal­iza­tion, let alone the dou­ble­think and de­nial-of-de­nial that char­ac­ter­izes anosog­nosics or hu­mans think­ing about poli­tics. In­spect­ing the AI’s thought pro­cesses seems to show that, in ac­cor­dance with your de­sign, the AI has no in­ten­tion to de­ceive you, and an ex­plicit goal of tel­ling you the truth. And in your ex­pe­rience so far, the AI has been, in­hu­manly, well-cal­ibrated; the AI has as­signed 99% cer­tainty on a cou­ple of hun­dred oc­ca­sions, and been wrong ex­actly twice that you know of.

Ar­guably, you now have far bet­ter rea­son to trust what the AI says to you, than to trust your own thoughts.

And now the AI tells you that it’s 99.9% sure—hav­ing seen it with its own cam­eras, and con­firmed from a hun­dred other sources—even though (it thinks) the hu­man brain is built to in­voke the ab­solute de­nial macro on it—that...


What’s the cra­ziest thing the AI could tell you, such that you would be will­ing to be­lieve that the AI was the sane one?

(Some of my own an­swers ap­pear in the com­ments.)