In the real world, Eliezer’s example simply doesn’t work.
In the real world you only hear about the results when they are published. The prior probability of the biased researcher publishing a positive result is higher than the prior probability of the unbiased researcher publishing a positive result.
The example only works if you are an omniscient spy who spies on absolutely all treatments. It’s true that an omniscient spy should just collate all the data regardless of the motivations of the researcher spied upon. However unless you are an omniscient spy you do need to take into account how an agent showing you new data went about gathering the data that they are showing you. Otherwise you are vulnerable to being “mushroomed” and updating yourself into beliefs that will cause you to lose.
In the real world, Eliezer’s example simply doesn’t work.
In the real world you only hear about the results when they are published. The prior probability of the biased researcher publishing a positive result is higher than the prior probability of the unbiased researcher publishing a positive result.
The example only works if you are an omniscient spy who spies on absolutely all treatments. It’s true that an omniscient spy should just collate all the data regardless of the motivations of the researcher spied upon. However unless you are an omniscient spy you do need to take into account how an agent showing you new data went about gathering the data that they are showing you. Otherwise you are vulnerable to being “mushroomed” and updating yourself into beliefs that will cause you to lose.