But what does this mean? Beliefs are about the world; goals are about what you would do with the world if you could rewrite it atom by atom. They’re totally different things; practically any goal is compatible with any belief, unless you’re infinitely convinced that some goal is literally impossible. Perhaps you’re saying that agents will dishonestly argue that the world is such that their goals will be easier to achieve than they are in fact? I can think of some situations where agents would find that useful. For myself, I care about honesty.
Notice that I did say “rational and honest.”
But what does this mean? Beliefs are about the world; goals are about what you would do with the world if you could rewrite it atom by atom. They’re totally different things; practically any goal is compatible with any belief, unless you’re infinitely convinced that some goal is literally impossible. Perhaps you’re saying that agents will dishonestly argue that the world is such that their goals will be easier to achieve than they are in fact? I can think of some situations where agents would find that useful. For myself, I care about honesty.
The way I read it ‘rational’ and ‘honest’ referred to the first clause of the sentence only.
For an example of an opinion consistent with an aim, consider a big tobacco sales exec who believes that cigarettes do not cause cancer.
We probably don’t actually disagree.