If someone could try to convince me that this experiment was not pointless and actually worth running for some reason, I would be interested to hear their arguments. Note that I’m more sympathetic to “this kind of experiment could be valuable if ran in the right environment”, and my skepticism is specifically about running it for chess.
I’ve been interested in the study of this question for a while. I agree this post has the flaws you point out, but I still find that it provides interesting evidence. If the result had been that Stockfish would have continued to win even with overwhelming material disadvantage, then this of course would have updated me some. I agree the current result is kind of close to the null result, but that’s fine. Also, it is much cheaper to run than almost all the other experiments in this space, and it’s good to encourage people to get started at all, even if it’s going to be somewhat streetlighty.
I’ve been interested in the study of this question for a while. I agree this post has the flaws you point out, but I still find that it provides interesting evidence. If the result had been that Stockfish would have continued to win even with overwhelming material disadvantage, then this of course would have updated me some. I agree the current result is kind of close to the null result, but that’s fine. Also, it is much cheaper to run than almost all the other experiments in this space, and it’s good to encourage people to get started at all, even if it’s going to be somewhat streetlighty.