It seems the ascription process is approximately “deduce an agent’s beliefs from their outputs”. This seems to have the same problem as “deduce an agent’s preferences from their outputs”, which I showed was not possible in general, even with simplicity.
So when dealing with non-perfectly rational agents, it seems you’ll have to put in the irrationality by hand. So it’s not so much “ascribing beliefs”, but “prescribing beliefs”: our interpretation determines what the agent believes. The fact that “This procedure wouldn’t capture the beliefs of a native Spanish speaker, or for someone who wasn’t answering questions honestly”, are just two examples of a much more universal problem.
It seems the ascription process is approximately “deduce an agent’s beliefs from their outputs”. This seems to have the same problem as “deduce an agent’s preferences from their outputs”, which I showed was not possible in general, even with simplicity.
So when dealing with non-perfectly rational agents, it seems you’ll have to put in the irrationality by hand. So it’s not so much “ascribing beliefs”, but “prescribing beliefs”: our interpretation determines what the agent believes. The fact that “This procedure wouldn’t capture the beliefs of a native Spanish speaker, or for someone who wasn’t answering questions honestly”, are just two examples of a much more universal problem.