How to notice being mind-hacked

Epistemic sta­tus: quite sure, but likely noth­ing new, I have not done the req­ui­site liter­a­ture search.

Hu­man mind is not de­signed with se­cu­rity in mind. It has some defenses against ba­sic ad­ver­saries that would have pre­vented our sur­vival as a species, but not much more than that. It is also nec­es­sar­ily open to ex­ter­nal in­fluences be­cause hu­mans are so­cial an­i­mals and co­op­er­a­tion is es­sen­tial for sur­vival. So, any se­cu­rity ex­pert would be hor­rified at how vuln­er­a­ble to ad­ver­sar­ial mind hack­ing hu­mans are. Hu­mans gen­er­ally do not like to ac­cept how easy we are to sway, and how of­ten it hap­pens to us, but we can definitely see other peo­ple be­ing eas­ily in­fluenced, and most of us aren’t spe­cial in terms of mind se­cu­rity.

Another com­mon term for it is “ma­nipu­la­tion,” but there is a slight differ­ence. Ma­nipu­la­tion gen­er­ally pre­sumes that the in­ter­ests of the ma­nipu­la­tor are detri­men­tal to the mind be­ing ma­nipu­lated. Mind hack­ing does not have to have this nega­tive con­no­ta­tion.

So, given that our minds are se­cu­rity sieves and we live in the world where in­fluenc­ing oth­ers (yet an­other term for mind hack­ing), and where we have cer­tainly been mind-hacked by oth­ers over and over again, how does one no­tice a hack (unau­tho­rized breach of mind se­cu­rity), whether when it is about to hap­pen, when in progress, and af­ter the fact? I am limit­ing the scope to just notic­ing. I am not im­ply­ing that one has to try to stop a mind hack in prepa­ra­tion or in progress, or try­ing to undo it af­ter it hap­pened. De­scrip­tive, not pre­scrip­tive.

Let’s start with a a few ob­vi­ous ex­am­ples.

Your friend, notic­ing your dis­tress, in­vites you to their church event, just to get your mind off things. A month later, you have con­verted to their faith, quote scrip­tures, be­lieve in sal­va­tion and ded­i­cate your life to spread­ing the gospel.

Or you come across a book, say, HPMoR or From AI to Zom­bies (I take par­tial credit/​blame for the lat­ter name), learn about ra­tio­nal­ity, get blown away by Eliezer’s ge­nius, and, next thing you know, you are at a lo­cal x-risk meetup wor­ry­ing about an un­al­igned AI ac­ci­den­tally pa­per-clip­ping the uni­verse and donat­ing 10% of your in­come to an EA cause.

Or you pick up a Siouxsie and the Ban­shees CD in a record store (back when CDs and record stores were a thing), and soon you are a part of the goth sub­cul­ture, death­hawk up ev­ery week­end, your care­fully crafted Rihanna mix­tape (an­other anachro­nism) gath­er­ing dust in the back of the bot­tom drawer.

Or maybe you end up at a kink munch, seem­ingly out of idle cu­ri­os­ity, then at a play party, then you dis­cover your sub­mis­sive side, end up dump­ing your vanilla part­ner and go on a sub frenzy and even­tu­ally set­tle as a slave to a Master/​Mistress.

Not all mind hacks are as strik­ing. But these some­what ex­treme, yet also main­stream ex­am­ples is a good place to start the anal­y­sis. Some salient fea­tures:

  • A glar­ing chasm be­tween your iden­tity be­fore and af­ter the event.

  • Ac­cep­tance of your cur­rent iden­tity and think­ing of your­self be­fore the event as im­ma­ture/​naive/​stupid/​un­en­light­ened.

  • Real­iza­tion that the you be­fore the event would likely be similarly dis­ap­prov­ing of the change that tran­spired and would have pre­vented it if they could an­ti­ci­pate it.

  • [What else?]

The above sug­gests how to no­tice the event post hoc (post hack?). The iden­tity dis­con­nect and the feel­ings around it are a tel­l­tale sign.

Notic­ing a hack­ing at­tempt or a hack in progress is prob­a­bly harder. When skil­lfully ex­e­cuted, it never rises to the con­scious level. You don’t nec­es­sar­ily con­sciously no­tice your iden­tity chang­ing. In­stead, you may be swept in the feel­ings of in­sight, be­ing wowed, en­light­ened, or the op­po­site, in­tense guilt, shame and re­morse, and of­ten some com­bi­na­tion of both. And even if we do rec­og­nize it for what it is, these same in­tense feel­ings can be too ad­dic­tive to break the spell, and we can crave them more and more, and ra­tio­nal­ize away what is hap­pen­ing. So, to pro­vi­sion­ally an­swer the ti­tle non-ques­tion, watch out for the mind-hack-as­so­ci­ated feel­ings.

What have been your ex­pe­riences with notic­ing be­ing mind hacked, in­ten­tion­ally or ac­ci­den­tally, or with do­ing it to oth­ers, whether on pur­pose or not?