Instrumental rationality is fairly easy to define when it’s about optimising your own UF. You seem to have the worry that optimising your own UF isn’t what you should be doing given the default meaning of “should”. Which is fair enough.
In a naive conception, a lot of people have a utility function that values the pleasure of eating highly sugared food and not eating a lot of green vegetables.
Self-modifying in a way that you get more pleasure from eating more healthy food seems naively useful but it’s changing the utility function. It’s unclear to what extent it makes sense to do things that do shift one’s utility function.
Every new experience may change your preferences. Seeking out new experiences will predictably have this effect. Openness to experience requires openness to seeing your preferences changing, even if you do not know in what direction. This has been going on since birth, as no-one is born with the preferences they will have as adults. There is a process by which they develop, which does not, or at least need not, stop on reaching adulthood.
Yes, experiences change preferences. This process is however not just random but can be guided. The question of how that process of changing the utility function can be addressed rationally is open.
Instrumental rationality is fairly easy to define when it’s about optimising your own UF. You seem to have the worry that optimising your own UF isn’t what you should be doing given the default meaning of “should”. Which is fair enough.
In a naive conception, a lot of people have a utility function that values the pleasure of eating highly sugared food and not eating a lot of green vegetables.
Self-modifying in a way that you get more pleasure from eating more healthy food seems naively useful but it’s changing the utility function. It’s unclear to what extent it makes sense to do things that do shift one’s utility function.
“I don’t like spinach, and I’m glad I don’t, because if I liked it i’d eat it, and I hate it.”
Every new experience may change your preferences. Seeking out new experiences will predictably have this effect. Openness to experience requires openness to seeing your preferences changing, even if you do not know in what direction. This has been going on since birth, as no-one is born with the preferences they will have as adults. There is a process by which they develop, which does not, or at least need not, stop on reaching adulthood.
Yes, experiences change preferences. This process is however not just random but can be guided. The question of how that process of changing the utility function can be addressed rationally is open.