This is connected to the Rebelling Within Nature post: have you considered that your criterion “you shouldn’t care about a value if it isn’t objective”, is another value that is particular to you as a human? A simple Paperclip Maximizer wouldn’t have the criterion “stop caring about paperclips if it turns out the goodness of paperclips isn’t written into the fabric of the universe”. (Nor would it have the criterion of respecting other agents’ moralities, another thing which you value.)
This is connected to the Rebelling Within Nature post: have you considered that your criterion “you shouldn’t care about a value if it isn’t objective”, is another value that is particular to you as a human? A simple Paperclip Maximizer wouldn’t have the criterion “stop caring about paperclips if it turns out the goodness of paperclips isn’t written into the fabric of the universe”. (Nor would it have the criterion of respecting other agents’ moralities, another thing which you value.)