I think Bob still doesn’t really need a two-part strategy in this case. Bob knows that Alice believes “time and space are relative”, so Bob believes this proposition, even though Bob doesn’t know the meaning of it. Bob doesn’t need any special-case rule to predict Alice. The best thing Bob can do in this case still seems like, predict Alice based off of Bob’s own beliefs.
(Perhaps you are arguing that Bob can’t believe something without knowing what that thing means? But to me this requires bringing in extra complexity which we don’t know how to handle anyway, since we don’t have a bayesian definition of “definition” to distinguish “Bob thinks X is true but doesn’t know what X means” from a mere “Bob thinks X is true”.)
A similar example would be an auto mechanic. You expect the mechanic to do things like pop the hood, get underneath the vehicle, grab a wrench, etc. However, you cannot predict which specific actions are useful for a given situation.
We could try to use a two-part model as you suggest, where we (1) maintain an incoherent-but-useful model of car-specific beliefs mechanics have, such as “wrenches are often needed”; (2) use the best of our own beliefs where that model doesn’t apply.
However, this doesn’t seem like it’s ever really necessary or like it saves processing power for bounded reasoners, because we also believe that “wrenches are sometimes useful”. This belief isn’t specific enough that we could reproduce the mechanic’s actions by acting on these beliefs; but, that’s fine, that’s just because we don’t know enough.
(Perhaps you have in mind a picture where we can’t let incoherent beliefs into our world-model—our limited understanding of Alice’s physics, or of the mechanic’s work, means that we want to maintain a separate, fully coherent world-model, and apply our limited understanding of expert knowledge only as a patch. If this is what you are getting at, this seems reasonable, so long as we can still count the whole resulting thing “my beliefs”—my beliefs, as a bounded agent, aren’t required to be one big coherent model.)
But, it does seem like there might be an example close to the one you spelled out. Perhaps when Alice says “X is relative”, Alice often starts doing an unfamiliar sort of math on the whiteboard. Bob has no idea how to interpret any of it as propositions—he can’t even properly divide it up into equations, to pay lip service to equations in the “X is true, but I don’t know what it means” sense I used above.
Then, it seems like Bob has to model Alice with a special-case “Alice starts writing the crazy math” model. Bob has some very basic beliefs about the math Alice is writing, such as “writing the letter Beta seems to be involved”, but these are clearly object-level beliefs about Alice’s behaviors, which Bob has to keep track of specifically. So in this situation it seems like Bob’s best model of Alice’s behavior doesn’t just follow from Bob’s own best model of what to do?
(So I end this comment on a somewhat uncertain note)
I think Bob still doesn’t really need a two-part strategy in this case. Bob knows that Alice believes “time and space are relative”, so Bob believes this proposition, even though Bob doesn’t know the meaning of it. Bob doesn’t need any special-case rule to predict Alice. The best thing Bob can do in this case still seems like, predict Alice based off of Bob’s own beliefs.
(Perhaps you are arguing that Bob can’t believe something without knowing what that thing means? But to me this requires bringing in extra complexity which we don’t know how to handle anyway, since we don’t have a bayesian definition of “definition” to distinguish “Bob thinks X is true but doesn’t know what X means” from a mere “Bob thinks X is true”.)
A similar example would be an auto mechanic. You expect the mechanic to do things like pop the hood, get underneath the vehicle, grab a wrench, etc. However, you cannot predict which specific actions are useful for a given situation.
We could try to use a two-part model as you suggest, where we (1) maintain an incoherent-but-useful model of car-specific beliefs mechanics have, such as “wrenches are often needed”; (2) use the best of our own beliefs where that model doesn’t apply.
However, this doesn’t seem like it’s ever really necessary or like it saves processing power for bounded reasoners, because we also believe that “wrenches are sometimes useful”. This belief isn’t specific enough that we could reproduce the mechanic’s actions by acting on these beliefs; but, that’s fine, that’s just because we don’t know enough.
(Perhaps you have in mind a picture where we can’t let incoherent beliefs into our world-model—our limited understanding of Alice’s physics, or of the mechanic’s work, means that we want to maintain a separate, fully coherent world-model, and apply our limited understanding of expert knowledge only as a patch. If this is what you are getting at, this seems reasonable, so long as we can still count the whole resulting thing “my beliefs”—my beliefs, as a bounded agent, aren’t required to be one big coherent model.)
But, it does seem like there might be an example close to the one you spelled out. Perhaps when Alice says “X is relative”, Alice often starts doing an unfamiliar sort of math on the whiteboard. Bob has no idea how to interpret any of it as propositions—he can’t even properly divide it up into equations, to pay lip service to equations in the “X is true, but I don’t know what it means” sense I used above.
Then, it seems like Bob has to model Alice with a special-case “Alice starts writing the crazy math” model. Bob has some very basic beliefs about the math Alice is writing, such as “writing the letter Beta seems to be involved”, but these are clearly object-level beliefs about Alice’s behaviors, which Bob has to keep track of specifically. So in this situation it seems like Bob’s best model of Alice’s behavior doesn’t just follow from Bob’s own best model of what to do?
(So I end this comment on a somewhat uncertain note)