I see lots of situations where let’s say, Mike is aware that either method A and method B that can be used to carry out a task.
Observational data shows that on average, system A outperforms system B, so seeing this, Mike decides to use system A.
But, the best ever result on the task was achieved with system B, and the conditions under which that was achieved could be easily replicated.
So really, Mike would be better off using system B and replicating those exact conditions—Mike could have cherry picked and recreated the best scenario, but made a decision based on averages instead.
Is there a term for this? And if not, what should the term be?