Verifying vNM-rationality requires an ontology

Re­sult
It is im­pos­si­ble to ver­ify that an agent is vNM-ra­tio­nal by ob­serv­ing its ac­tions with­out ac­cess to the do­main of its util­ity func­tion.


Mo­ti­va­tion
Alphonso and Beatriz both go the mar­ket to buy fruit.
Alphonso prefers grapes to or­anges.
He fills his bas­ket with grapes and pays for them.

Beatriz care­fully picks through the fruit and pur­chases some or­anges and some grapes.
Cal­listo ar­rives with a pack­age of grapes.

”Say, Beatriz, would you like to trade some of your or­anges for this pack­age of grapes?” Cal­listo offers.

”Gladly.” Beatriz replies, ex­chang­ing some of her or­anges for the grapes.

A few mo­ments later, Alphonso no­tices Beatriz giv­ing Deion some grapes in ex­change for some or­anges.

”You are act­ing ir­ra­tionally, Beatriz!” Alphonso ex­claims. “Your un­sta­ble prefer­ence be­tween or­anges and grapes makes it pos­si­ble for a mal­i­cious agent to ex­ploit you and ex­haust your en­tire gro­cery bud­get!

”Ah, but I am act­ing ra­tio­nally.” Beatriz replied with a smile. “I pre­fer fruit that is fresh enough to last more than seven days. Thus, I trade away fruit that will spoil be­fore that time.”

Ex­pla­na­tion
Con­sider an agent A.
We are in­ter­ested in ver­ify­ing whether or not A is vNM-ra­tio­nal.
How­ever, we are only able to ob­serve A’s de­ci­sions with­out any ac­cess to the do­main of A’s util­ity func­tion.

Without this ac­cess, it is im­pos­si­ble to dis­t­in­guish be­tween vNM-ir­ra­tional choices (i.e. choices that vi­o­late one of the ax­ioms of vNM-ra­tio­nal­ity) and choices that are vNM-ra­tio­nal but made un­der an un­ex­pected on­tol­ogy.


In other words, we need to know how A per­ceives out­comes of the world be­fore we can ver­ify that A’s prefer­ences over those out­comes are vNM-ra­tio­nal.