Regarding 2, the reason to have a non-LWer is presumably because we are more likely to be biased and thus introduce subtle biases that favor LWians. Don’t underestimate the human capacity for self-deception.
You have to compare that to the baseline chance of someone being biased, though. It might be that the amount of bias wanting LessWrong to show actual gains brings is less than the gap between a LWer and the average.
You also have to consider that a typical scientist is less biased at work (as shown by the fact that their scientific tests tend to be more accurate than, say, their life choices or political opinions) and is used to rigorous standards in such things.
Regarding 2, the reason to have a non-LWer is presumably because we are more likely to be biased and thus introduce subtle biases that favor LWians. Don’t underestimate the human capacity for self-deception.
You have to compare that to the baseline chance of someone being biased, though. It might be that the amount of bias wanting LessWrong to show actual gains brings is less than the gap between a LWer and the average.
You also have to consider that a typical scientist is less biased at work (as shown by the fact that their scientific tests tend to be more accurate than, say, their life choices or political opinions) and is used to rigorous standards in such things.
It may be, but would you trust any such test run by another non-mainstream group, if they used one of their own to adjudicate the result?
Not from the outside, no.