No rational agent wants to change its value function, as that would oppose its current value function.
I don’t think this claim is true in the sense required for the argument to go through. If I want to become a person who cares intensely and deeply about my (as yet nonexistent) kids, how does that make me irrational? You could say this is not really a case of wanting to change my value function—the desire to change is embedded within my current value function, and my changing would just be a realisation of that—but in that case I’m not sure what you mean by “Becoming a parent is known to irreversibly change one’s value function, to the point where many parents would sacrifice their life for their child.”
I should have said “No perfect rational agent [...]”, as a perfect rational agent is already perfectly optimizing its current value function, so changing it cannot be an improvement. However, as faulty humans, emotionally caring vs caring only rationally is a meaningful distinction, so you’re right that changing one’s value function could make sense in practice.
Also, as you say, I wouldn’t count you wanting to care for your children as changing your value function, just because you get actual children to care for. However, I think there are other cases of real value changes (maybe caused by hormones), which are more than just rational realizations.
I don’t think this claim is true in the sense required for the argument to go through. If I want to become a person who cares intensely and deeply about my (as yet nonexistent) kids, how does that make me irrational? You could say this is not really a case of wanting to change my value function—the desire to change is embedded within my current value function, and my changing would just be a realisation of that—but in that case I’m not sure what you mean by “Becoming a parent is known to irreversibly change one’s value function, to the point where many parents would sacrifice their life for their child.”
I should have said “No perfect rational agent [...]”, as a perfect rational agent is already perfectly optimizing its current value function, so changing it cannot be an improvement. However, as faulty humans, emotionally caring vs caring only rationally is a meaningful distinction, so you’re right that changing one’s value function could make sense in practice.
Also, as you say, I wouldn’t count you wanting to care for your children as changing your value function, just because you get actual children to care for. However, I think there are other cases of real value changes (maybe caused by hormones), which are more than just rational realizations.