If you don’t fully trust that agent, though, then it seems very tricky to reason about how much you should defer to them, because they may be manipulating you heavily. In such cases the approach that seems most robust is to diversify worldviews using a meta-rationality strategy which includes some strong principles.
This doesn’t seem to follow. Why wouldn’t the ‘strong principles’ also be a product of heavy manipulation?
Regardless of how much harder they may be to manipulate, they can never be invulnerable. Which implies that given enough time, all principles, even the strongest, are subject to change.
This doesn’t seem to follow. Why wouldn’t the ‘strong principles’ also be a product of heavy manipulation?
Strong principles tend to be harder to manipulate, because:
a) Strong principles tend to be simple and clear; there’s not much room for cherrypicking them to produce certain outcomes.
b) Principle-driven actions are less dependent on your specific beliefs.
Regardless of how much harder they may be to manipulate, they can never be invulnerable. Which implies that given enough time, all principles, even the strongest, are subject to change.