How valuable was each experiment? If you make a theory after ten experiments, then you could design the next ten experiments to be very specific to the theory: if the theory is right, they come out one way, and they should come out a very different way if the theory is wrong.
It’s like writing test-cases for software: once you know what exactly you’re testing, you can write test cases that aim at any potential weak spots, in a deliberate and targeted attempt to break it. So if we’re assuming that these scientists are actual people (rather than really hypothetical abstractions), I would give more credence to the guy who did ten experiments, formulated a theory, and did ten more experiments, iff those later ten experiments look like they were designed to give new information and really stress-test the theory.
If we’re talking about the exact same 20 experiments, then I would generally favor doing maybe 13 experiments, making a theory, and then doing the other 7, to avoid overfitting. Or split the experiments up into two sets of ten, and have two scientists each look at ten of the experiments, make a theory, then test it with the other ten. This kind of thinking would have killed Ptolemy’s theory of epicycles, which is a classic case of overcomplicating the theory to match your observations.
I know that’s hardly a formal answer, but I think the original question was oversimplifying.
How valuable was each experiment? If you make a theory after ten experiments, then you could design the next ten experiments to be very specific to the theory: if the theory is right, they come out one way, and they should come out a very different way if the theory is wrong.
It’s like writing test-cases for software: once you know what exactly you’re testing, you can write test cases that aim at any potential weak spots, in a deliberate and targeted attempt to break it. So if we’re assuming that these scientists are actual people (rather than really hypothetical abstractions), I would give more credence to the guy who did ten experiments, formulated a theory, and did ten more experiments, iff those later ten experiments look like they were designed to give new information and really stress-test the theory.
If we’re talking about the exact same 20 experiments, then I would generally favor doing maybe 13 experiments, making a theory, and then doing the other 7, to avoid overfitting. Or split the experiments up into two sets of ten, and have two scientists each look at ten of the experiments, make a theory, then test it with the other ten. This kind of thinking would have killed Ptolemy’s theory of epicycles, which is a classic case of overcomplicating the theory to match your observations.
I know that’s hardly a formal answer, but I think the original question was oversimplifying.