This is the kind of issue that being really specific can be used to resolve. If the Reporters were specific about what they were trying to do in their reporting, the Audience would not have used their reports. If this is unfeasible, the Audience could still specifically ask the Reporters to clarify whether they think she will be able to figure out the specific things she is trying to figure out using their reports. If they mislead her here, Audience can credibly claim that either she was lied to or that the reporters are not competent enough to be relied on for information. Pragmatically, both these possibilities imply a course-of-action that involved not using the reporters as reporters.
To illustrate in detail:
So, too, imagine each of our possible Reporters as a person: loyal, responsible—and, entirely coincidentally, the supplier of a good that Audience’s careful plans call for in proportion to the value of P(X=4).
In this context, the audience has an explicit goal of estimating how frequently X=4 occurs. Keeping this in mind, the problem the audience is having can be understood in the context of interacting with reports with misaligned goals.
Sure, I’m not a perfect program free from all bias, but everything I said was true—every outcome I reported corresponded to one of the Xi. You can’t call that misleading!”
Reporter 2′s goal seems to be “report values that correspond to one of the Xi.” This is misaligned with a goal associated with estimating the distribution of X.
I told you the truth, the whole truth, and nothing but the truth: everything I saw, I reported. When I said an outcome was a oneorfour, it actually was a oneorfour. Perhaps you have a different category system, such that what I think of as a ‘oneorfour’, appears to you to be any of several completely different outcomes, which you think my ‘oneorfour’ concept is conflating.
Reporter 3′s goal seems to be “report values but conflate ones and fours into a ‘oneorfour’ category” (in the actual programming, this is a misrepresentation of course. Reporting a one or a four as a one and a four isn’t the same thing). This is misaligned with the goal of figuring out how frequently fours are specifically.
If Reporter 2 or Reporter 3 had explicitly specified what they were trying to do, the Audience wouldn’t have used them as an information source.
If Reporter 2 had told Audience that he believed Alice could determine probabilities just from his reporting, he would be lying, or too incompetent to trust.
If Reporter 3 had told Audience that he believed Alice could distinguish the probability of fours separate from the probabilities of ones, he would be lying, or too incompetent to trust.
This is the kind of issue that being really specific can be used to resolve. If the Reporters were specific about what they were trying to do in their reporting, the Audience would not have used their reports. If this is unfeasible, the Audience could still specifically ask the Reporters to clarify whether they think she will be able to figure out the specific things she is trying to figure out using their reports. If they mislead her here, Audience can credibly claim that either she was lied to or that the reporters are not competent enough to be relied on for information. Pragmatically, both these possibilities imply a course-of-action that involved not using the reporters as reporters.
To illustrate in detail:
In this context, the audience has an explicit goal of estimating how frequently X=4 occurs. Keeping this in mind, the problem the audience is having can be understood in the context of interacting with reports with misaligned goals.
Reporter 2′s goal seems to be “report values that correspond to one of the Xi.” This is misaligned with a goal associated with estimating the distribution of X.
Reporter 3′s goal seems to be “report values but conflate ones and fours into a ‘oneorfour’ category” (in the actual programming, this is a misrepresentation of course. Reporting a one or a four as a one and a four isn’t the same thing). This is misaligned with the goal of figuring out how frequently fours are specifically.
If Reporter 2 or Reporter 3 had explicitly specified what they were trying to do, the Audience wouldn’t have used them as an information source.
If Reporter 2 had told Audience that he believed Alice could determine probabilities just from his reporting, he would be lying, or too incompetent to trust.
If Reporter 3 had told Audience that he believed Alice could distinguish the probability of fours separate from the probabilities of ones, he would be lying, or too incompetent to trust.