Why do you think that quoted example would be in the category “hardcoding formerly instrumental goals to be terminal goals”, rather than the category “hardcoding terminal goals to prevent them from changing”?
Like, imagine Hitler getting the godlike powers of knowledge and self-control. If he gets the perfect knowledge of economy, sociology and psychology first, he could go like: “Oh, now I realize that the things I blamed on the Jews are actually caused by something else. How embarrassing. No more anti-semitism, but I better erase everyone’s memory first.”
This suggested (and I think this is in fact plausible) that Hitler’s hate for Jews was not a terminal value of his, but that he mistakenly believed they were causing various bad outcomes via conspiracy, so he concluded they were evil. Then hardcoding this dislike for Jews would be transforming an instrumental value into a terminal value.
I also think that “hardcoding terminal goals to prevent them from changing” doesn’t make a lot of sense, since terminal goals are already pretty much as hardcoded as the brain wetware allows. E.g., if you liked lasagna in your 20s, there is a high probability that you still like it in your 60s.
Why do you think that quoted example would be in the category “hardcoding formerly instrumental goals to be terminal goals”, rather than the category “hardcoding terminal goals to prevent them from changing”?
Look at the previous paragraph:
This suggested (and I think this is in fact plausible) that Hitler’s hate for Jews was not a terminal value of his, but that he mistakenly believed they were causing various bad outcomes via conspiracy, so he concluded they were evil. Then hardcoding this dislike for Jews would be transforming an instrumental value into a terminal value.
I also think that “hardcoding terminal goals to prevent them from changing” doesn’t make a lot of sense, since terminal goals are already pretty much as hardcoded as the brain wetware allows. E.g., if you liked lasagna in your 20s, there is a high probability that you still like it in your 60s.