Ontological Crisis in Humans

Imag­ine a robot that was de­signed to find and col­lect spare change around its owner’s house. It had a world model where macro­scopic ev­ery­day ob­jects are on­tolog­i­cally prim­i­tive and ruled by high-school-like physics and (for hu­mans and their pets) rudi­men­tary psy­chol­ogy and an­i­mal be­hav­ior. Its goals were ex­pressed as a util­ity func­tion over this world model, which was suffi­cient for its de­signed pur­pose. All went well un­til one day, a prankster de­cided to “up­grade” the robot’s world model to be based on mod­ern par­ti­cle physics. This un­for­tu­nately caused the robot’s util­ity func­tion to in­stantly throw a do­main er­ror ex­cep­tion (since its in­puts are no longer the ex­pected list of macro­scopic ob­jects and as­so­ci­ated prop­er­ties like shape and color), thus crash­ing the con­trol­ling AI.

Ac­cord­ing to Peter de Blanc, who used the phrase “on­tolog­i­cal crisis” to de­scribe this kind of prob­lem,

Hu­man be­ings also con­front on­tolog­i­cal crises. We should find out what cog­ni­tive al­gorithms hu­mans use to solve the same prob­lems de­scribed in this pa­per. If we wish to build agents that max­i­mize hu­man val­ues, this may be aided by know­ing how hu­mans re-in­ter­pret their val­ues in new on­tolo­gies.

I re­cently re­al­ized that a cou­ple of prob­lems that I’ve been think­ing over (the na­ture of self­ish­ness and the na­ture of pain/​plea­sure/​suffer­ing/​hap­piness) can be con­sid­ered in­stances of on­tolog­i­cal crises in hu­mans (al­though I’m not so sure we nec­es­sar­ily have the cog­ni­tive al­gorithms to solve them). I started think­ing in this di­rec­tion af­ter writ­ing this com­ment:

This for­mu­la­tion or var­i­ant of TDT re­quires that be­fore a de­ci­sion prob­lem is handed to it, the world is di­vided into the agent it­self (X), other agents (Y), and “dumb mat­ter” (G). I think this is mis­guided, since the world doesn’t re­ally di­vide cleanly into these 3 parts.

What struck me is that even though the world doesn’t di­vide cleanly into these 3 parts, our mod­els of the world ac­tu­ally do. In the world mod­els that we hu­mans use on a day to day ba­sis, and over which our util­ity func­tions seem to be defined (to the ex­tent that we can be said to have util­ity func­tions at all), we do take the Self, Other Peo­ple, and var­i­ous Dumb Mat­ter to be on­tolog­i­cally prim­i­tive en­tities. Our world mod­els, like the coin col­lect­ing robot’s, con­sist of these macro­scopic ob­jects ruled by a hodge­podge of heuris­tics and pre­dic­tion al­gorithms, rather than micro­scopic par­ti­cles gov­erned by a co­her­ent set of laws of physics.

For ex­am­ple, the amount of pain some­one is ex­pe­rienc­ing doesn’t seem to ex­ist in the real world as an XML tag at­tached to some “per­son en­tity”, but that’s pretty much how our mod­els of the world work, and per­haps more im­por­tantly, that’s what our util­ity func­tions ex­pect their in­puts to look like (as op­posed to, say, a list of par­ti­cles and their po­si­tions and ve­loc­i­ties). Similarly, a hu­man can be self­ish just by treat­ing the ob­ject la­beled “SELF” in its world model differ­ently from other ob­jects, whereas an AI with a world model con­sist­ing of micro­scopic par­ti­cles would need to some­how in­herit or learn a de­tailed de­scrip­tion of it­self in or­der to be self­ish.

To fully con­front the on­tolog­i­cal crisis that we face, we would have to up­grade our world model to be based on ac­tual physics, and si­mul­ta­neously trans­late our util­ity func­tions so that their do­main is the set of pos­si­ble states of the new model. We cur­rently have lit­tle idea how to ac­com­plish this, and in­stead what we do in prac­tice is, as far as I can tell, keep our on­tolo­gies in­tact and util­ity func­tions un­changed, but just add some new heuris­tics that in cer­tain limited cir­cum­stances call out to new physics for­mu­las to bet­ter up­date/​ex­trap­o­late our mod­els. This is ac­tu­ally rather clever, be­cause it lets us make use of up­dated un­der­stand­ings of physics with­out ever hav­ing to, for in­stance, de­cide ex­actly what pat­terns of par­ti­cle move­ments con­sti­tute pain or plea­sure, or what pat­terns con­sti­tute one­self. Nev­er­the­less, this ap­proach hardly seems ca­pa­ble of be­ing ex­tended to work in a fu­ture where many peo­ple may have non­tra­di­tional mind ar­chi­tec­tures, or have a zillion copies of them­selves run­ning on all kinds of strange sub­strates, or be merged into amor­phous group minds with no clear bound­aries be­tween in­di­vi­d­u­als.

By the way, I think nihilism of­ten gets short changed around here. Given that we do not ac­tu­ally have at hand a solu­tion to on­tolog­i­cal crises in gen­eral or to the spe­cific crisis that we face, what’s wrong with say­ing that the solu­tion set may just be null? Given that evolu­tion doesn’t con­sti­tute a par­tic­u­larly benev­olent and far­sighted de­signer, per­haps we may not be able to do much bet­ter than that poor spare-change col­lect­ing robot? If Eliezer is wor­ried that ac­tual AIs fac­ing ac­tual on­tolog­i­cal crises could do worse than just crash, should we be very san­guine that for hu­mans ev­ery­thing must “add up to moral nor­mal­ity”?

To ex­pand a bit more on this pos­si­bil­ity, many peo­ple have an aver­sion against moral ar­bi­trari­ness, so we need at a min­i­mum a util­ity trans­la­tion scheme that’s prin­ci­pled enough to pass that filter. But our ex­ist­ing world mod­els are a hodge­podge put to­gether by evolu­tion so there may not be any such suffi­ciently prin­ci­pled scheme, which (if other ap­proaches to solv­ing moral philos­o­phy also don’t pan out) would leave us with le­gi­t­i­mate feel­ings of “ex­is­ten­tial angst” and nihilism. One could per­haps still ar­gue that any cur­rent such feel­ings are pre­ma­ture, but maybe some peo­ple have stronger in­tu­itions than oth­ers that these prob­lems are un­solv­able?

Do we have any ex­am­ples of hu­mans suc­cess­fully nav­i­gat­ing an on­tolog­i­cal crisis? The LessWrong Wiki men­tions loss of faith in God:

In the hu­man con­text, a clear ex­am­ple of an on­tolog­i­cal crisis is a be­liever’s loss of faith in God. Their mo­ti­va­tions and goals, com­ing from a very spe­cific view of life sud­denly be­come ob­so­lete and maybe even non­sense in the face of this new con­figu­ra­tion. The per­son will then ex­pe­rience a deep crisis and go through the psy­cholog­i­cal task of re­con­struct­ing its set of prefer­ences ac­cord­ing the new world view.

But I don’t think loss of faith in God ac­tu­ally con­sti­tutes an on­tolog­i­cal crisis, or if it does, cer­tainly not a very se­vere one. An on­tol­ogy con­sist­ing of Gods, Self, Other Peo­ple, and Dumb Mat­ter just isn’t very differ­ent from one con­sist­ing of Self, Other Peo­ple, and Dumb Mat­ter (the lat­ter could just be con­sid­ered a spe­cial case of the former with quan­tity of Gods be­ing 0), es­pe­cially when you com­pare ei­ther on­tol­ogy to one made of micro­scopic par­ti­cles or even less fa­mil­iar en­tities.

But to end on a more pos­i­tive note, re­al­iz­ing that seem­ingly un­re­lated prob­lems are ac­tu­ally in­stances of a more gen­eral prob­lem gives some hope that by “go­ing meta” we can find a solu­tion to all of these prob­lems at once. Maybe we can solve many eth­i­cal prob­lems si­mul­ta­neously by dis­cov­er­ing some generic al­gorithm that can be used by an agent to tran­si­tion from any on­tol­ogy to an­other?

(Note that I’m not say­ing this is the right way to un­der­stand one’s real prefer­ences/​moral­ity, but just draw­ing at­ten­tion to it as a pos­si­ble al­ter­na­tive to other more “ob­ject level” or “purely philo­soph­i­cal” ap­proaches. See also this pre­vi­ous dis­cus­sion, which I re­called af­ter writ­ing most of the above.)