I should have said “No perfect rational agent [...]”, as a perfect rational agent is already perfectly optimizing its current value function, so changing it cannot be an improvement. However, as faulty humans, emotionally caring vs caring only rationally is a meaningful distinction, so you’re right that changing one’s value function could make sense in practice.
Also, as you say, I wouldn’t count you wanting to care for your children as changing your value function, just because you get actual children to care for. However, I think there are other cases of real value changes (maybe caused by hormones), which are more than just rational realizations.
I should have said “No perfect rational agent [...]”, as a perfect rational agent is already perfectly optimizing its current value function, so changing it cannot be an improvement. However, as faulty humans, emotionally caring vs caring only rationally is a meaningful distinction, so you’re right that changing one’s value function could make sense in practice.
Also, as you say, I wouldn’t count you wanting to care for your children as changing your value function, just because you get actual children to care for. However, I think there are other cases of real value changes (maybe caused by hormones), which are more than just rational realizations.