Mysteries, identity, and preferences over non-rewards

Note: work­ing on a re­search agenda, hence the large amount of small in­di­vi­d­ual posts, to have things to link to in the main doc­u­ments.

I’ve given ex­am­ples of prefer­ences over non-re­wards.

But those ex­am­ples are easy to dis­miss as ir­ra­tional or im­pos­ing on oth­ers, as they in­volved re­li­gion, or things like prefer­ences over other peo­ple know­ing the truth.

But if we think in terms of prefer­ences over per­sonal iden­tity, we have far more wide­spread and nat­u­ral ex­am­ples of this.

One small ex­am­ple: in most mur­der-mys­ter­ies, I’d pre­fer not to know the iden­tity of the kil­ler at the be­gin­ning of the show. If I’m tak­ing part in a mur­der mys­tery din­ner, I’d definitely not want too. If I’m play­ing games of hid­den in­for­ma­tion for fun, then I would not want to know all the hid­den in­for­ma­tion—or else that makes this pointless. Lots of peo­ple en­joy watch­ing sport events live; much less peo­ple re-watch ran­dom past sport­ing events whose re­sult they already know. Eliezer talks about the joy in sci­en­tific dis­cov­ery.

So prefer­ences over non-re­wards are quite nat­u­ral and com­mon.

He­donism solu­tion?

Now, you could say you don’t have a “prefer­ences over know­ing/​not know­ing some­thing”, but that you “en­joy the ex­pe­rience of learn­ing”, and thus re­duce the whole thing to he­do­nism. Eliezer ar­gues against this, claiming to have prefer­ences that don’t re­duce to he­do­nism (prefer­ences I share, in­ci­den­tally).

But even fully giv­ing in to he­do­nism only par­tially solves the prob­lem; yes, we can model some­one’s faith as say­ing they en­joy the ex­pe­rience of be­lief, and mur­der-mys­tery watch­ers en­joy the ex­pe­rience of see­ing in­tri­cate puz­zles solved. But this doesn’t solve the prob­lem that, to max­imise he­do­nic ex­pe­rience, we still have to con­trol what knowl­edge the hu­man has.