That’s not quite what I meant. It is not the experimenter’s thoughts that I am uncomfortable with- it is the collection of possible experimental outcomes.
I will try to illustrate with an example. Let us say that I toss a coin either (i) two times, or (ii) until it comes up heads. In the first case, the possible outcomes are HH, HT, TH, or TT; in the second case, they are H, TH, TTH, TTTH, TTTTH, etc. It isn’t obvious to me that a TH outcome has the same meaning in both cases. If, for instance, we were not talking about likelihood and instead decided to measure something else, e.g. the portion of tosses landing on heads, this wouldn’t be the case; in scenario (i), the expected portion of tosses landing on heads is 1⁄4 + .5/4 + .5/4 + 0⁄4 = .5, but in scenario (ii), it would be 1⁄2 + .5/4 + (1/3)/8 + .25/16 + … = log(2); i.e. a little under .7.
I think in this case, we are assuming total and honest reporting of results (including publication); otherwise, we would be back to the story of filtered evidence. Therefore, the publication is not a result of the plans—it was going to happen in either case.
Thanks, I understood the mathematical point but was wondering if there is any practical significance since it seems in the real world that we cannot make such an assumption, and that in the real world we should trust the results of the two researchers differently (since the one researcher likely published no matter what, whereas the second probably only published the experiments which came out favorably (even if he didn’t publish false information)).
What is the practical import of this idea? In the real world with all of people’s biases shouldn’t we distinguish between the two researchers as a general heuristic for good research standards?
(If this is addressed in a different post on this site feel free to point me there since I have not read the majority of the site)
That’s not quite what I meant. It is not the experimenter’s thoughts that I am uncomfortable with- it is the collection of possible experimental outcomes.
I will try to illustrate with an example. Let us say that I toss a coin either (i) two times, or (ii) until it comes up heads. In the first case, the possible outcomes are HH, HT, TH, or TT; in the second case, they are H, TH, TTH, TTTH, TTTTH, etc. It isn’t obvious to me that a TH outcome has the same meaning in both cases. If, for instance, we were not talking about likelihood and instead decided to measure something else, e.g. the portion of tosses landing on heads, this wouldn’t be the case; in scenario (i), the expected portion of tosses landing on heads is 1⁄4 + .5/4 + .5/4 + 0⁄4 = .5, but in scenario (ii), it would be 1⁄2 + .5/4 + (1/3)/8 + .25/16 + … = log(2); i.e. a little under .7.
The TH outcome tells you the same thing about the coin because the coin does not know what your plans were like.
I’m convinced. Having though about this a little more, I think I see the model you are working under, and it does make a good deal of intuitive sense.
Does the publication of the result tell you the same thing, since the fact that it was published is a result of the plans?
I think in this case, we are assuming total and honest reporting of results (including publication); otherwise, we would be back to the story of filtered evidence. Therefore, the publication is not a result of the plans—it was going to happen in either case.
Thanks, I understood the mathematical point but was wondering if there is any practical significance since it seems in the real world that we cannot make such an assumption, and that in the real world we should trust the results of the two researchers differently (since the one researcher likely published no matter what, whereas the second probably only published the experiments which came out favorably (even if he didn’t publish false information)). What is the practical import of this idea? In the real world with all of people’s biases shouldn’t we distinguish between the two researchers as a general heuristic for good research standards?
(If this is addressed in a different post on this site feel free to point me there since I have not read the majority of the site)