Woops, looks like I may have shot myself in the foot. The same way argument screens off authority, the actual experiment that was run screens off the intentions of the researcher.
Efficacy of the drug → Results of the experiment ← Bias of the researcher
Efficacy, Bias → Results of the experiment → Our analysis of the efficacy of the drug
It gets interesting when the pebblesorters turn on a correctly functioning FAI, which starts telling them that they should build a pile of 108301 and legislative bodies spend the next decade debating whether or not it is in fact a correct pile. “How does this AI know better anyway? That looks new and strange.” “That doesn’t sound correct to me at all. You’d have to be crazy to build 108301. It’s so different from 2029! It’s a slippery slope to 256!” And so on.
This really is a fantastic parable—it shows off perhaps a dozen different aspects of the forrest we were missing for the trees.