But suppose that you are perfectly above all social pressure, and you happen to not like beer at the moment. Is there a good reason for you to choose the reprogramming anyway? Sure, if you do, you will be happy that you did. But if you won’t, you won’t miss it. (Maybe curiosity is the remaining argument for the change? But without the social pressure, there is no reason to be curious about this specific thing.) So it seems to me that there are two different reflectively stable outcomes, and it depends on history where you end up.
In practice, this path dependency thing is indeed important, but it has a lot to do with our tendency to get lost and fail to find coherence.
For example, instead of beer what about heroin? Needles are icky, but boy will that change if you give it a shot! What happens in the longer term isn’t so simple as “more happy” though, and especially when the effects come from exogenous chemicals, we can’t really trust our initial pleasure to cache out in anything real.
This can make heroin very risky because people will often fail to learn that heroin injections are yucky again, but that is where the road coheres to. I don’t have any experience with heroin, but I have tried legally prescribed opioids a couple times and went through this arc. After the first time I couldn’t stop thinking about it for a month because it felt so good. Eventually though, my brain kinda recognized that this is not actually a good thing, I don’t actually want it, and when I eventually tried it again “just because” it wasn’t even enjoyable.
Beer and coffee are a lot more subtle and have context dependent social stuff going on, but “If you do you’ll be glad you did” is far from a sure thing.
We could even go further; with superior self-modification skills I could give myself a completely arbitrary preference, for example a sexual fetish for triangles. It seems silly now, but after modifying myself, and decorating my palace with paintings of triangles, I would probably be happy that I did it. I may even feel a little curious right now about what such absurd situation would be like. So should I modify myself that way?
I mean… I kinda vote “yes”. Because curiosity is important, the things you learn from experience are important, and this is a relatively harmless example to experiment with.
In my experience though, arbitrary modifications like this aren’t very stable. If there isn’t any actual value being delivered, and you don’t try to set up a labyrinth of motivations to not-look, people tend to learn that triangles aren’t so exciting as they had tricked themselves into believing.
All new knowledge is self-modification, but not all self-modification is new (external) knowledge. You can also self-modify by resolving internal tensions. Or by changing a random connection in your brain.
A railroad spike to the brain will indeed indeed modify you, and is not well described as “learning new things!”. But resolving internal tensions usually is. And the railroad spike to the brain/random connection change is generally well described as losing things you have learned, or learning (expected) falsehoods.
Also, if you obtain new knowledge in different order, the later information gets interpreted in the light of the former.
This is another “probably in practice, but only because we don’t reach coherence” things. Maybe you consider the latter in terms of the former without going back to reinterpret the former in terms of the latter, but Bayes doesn’t justify this failure to propagate updates with any sort of path dependence.
The all-knowing Hitler would know that his original reasons for hating Jews are no longer valid, but he might retain an aesthetic preference for doing so, for example because the very emotion of feeling superior to someone feels enjoyable.
My point is that what we think of as inscrutable aesthetic preferences are built upon implicit beliefs about the world, and updating the underlying structure changes the aesthetics. Hitler may like to feel superior, but his potential superiority is itself a fact about reality that he could update on.
What happens when you sit with the question “Are you superior?”?
There’s often an impulse to flinch away, and refuse to update, but when you do, things change.
Thanks for the detailed response
In practice, this path dependency thing is indeed important, but it has a lot to do with our tendency to get lost and fail to find coherence.
For example, instead of beer what about heroin? Needles are icky, but boy will that change if you give it a shot! What happens in the longer term isn’t so simple as “more happy” though, and especially when the effects come from exogenous chemicals, we can’t really trust our initial pleasure to cache out in anything real.
This can make heroin very risky because people will often fail to learn that heroin injections are yucky again, but that is where the road coheres to. I don’t have any experience with heroin, but I have tried legally prescribed opioids a couple times and went through this arc. After the first time I couldn’t stop thinking about it for a month because it felt so good. Eventually though, my brain kinda recognized that this is not actually a good thing, I don’t actually want it, and when I eventually tried it again “just because” it wasn’t even enjoyable.
Beer and coffee are a lot more subtle and have context dependent social stuff going on, but “If you do you’ll be glad you did” is far from a sure thing.
I mean… I kinda vote “yes”. Because curiosity is important, the things you learn from experience are important, and this is a relatively harmless example to experiment with.
In my experience though, arbitrary modifications like this aren’t very stable. If there isn’t any actual value being delivered, and you don’t try to set up a labyrinth of motivations to not-look, people tend to learn that triangles aren’t so exciting as they had tricked themselves into believing.
A railroad spike to the brain will indeed indeed modify you, and is not well described as “learning new things!”. But resolving internal tensions usually is. And the railroad spike to the brain/random connection change is generally well described as losing things you have learned, or learning (expected) falsehoods.
This is another “probably in practice, but only because we don’t reach coherence” things. Maybe you consider the latter in terms of the former without going back to reinterpret the former in terms of the latter, but Bayes doesn’t justify this failure to propagate updates with any sort of path dependence.
My point is that what we think of as inscrutable aesthetic preferences are built upon implicit beliefs about the world, and updating the underlying structure changes the aesthetics. Hitler may like to feel superior, but his potential superiority is itself a fact about reality that he could update on.
What happens when you sit with the question “Are you superior?”?
There’s often an impulse to flinch away, and refuse to update, but when you do, things change.