Well… different ranking outcomes (different sides of the coin) are possible. Just that the interpretation will always be “don’t reject the null hypothesis” but yeah. :)
Either way, my overall reaction to your post is “yuck” (not your post itself! That I upvoted. I mean the whole situation… That a relatively standard statistical test could allow this sort of madness. I mean, I know frequentist stats isn’t the Bayesian way, but that relatively standard methods in it can be this pathological does not at all give me warm fuzzies)
I concur with your “yuck”, but would phrase it slightly differently. The specific type of statistical test applied, plus the number of samples taken, has the effect, as Cyan said, of guaranteeing the results that the authors wanted. Note that, more generally, the fact that the authors chose to phrase their analysis so that accepting the null hypothesis was the result they wanted plus choosing a nonparametric statistical test, which is always weaker than a parametric one is in and of itself suspicious. If they had had enough samples so that it would be theoretically possible for the null hypothesis to be rejected (say if they had taken more samples) but they had still wanted the null result and they had still chosen a nonparametric test I would still be suspicious. As Cyan said, the nonparametric tests throw away most of the information.
More of a double-headed coin situation, actually.
Well… different ranking outcomes (different sides of the coin) are possible. Just that the interpretation will always be “don’t reject the null hypothesis” but yeah. :)
Either way, my overall reaction to your post is “yuck” (not your post itself! That I upvoted. I mean the whole situation… That a relatively standard statistical test could allow this sort of madness. I mean, I know frequentist stats isn’t the Bayesian way, but that relatively standard methods in it can be this pathological does not at all give me warm fuzzies)
I concur with your “yuck”, but would phrase it slightly differently. The specific type of statistical test applied, plus the number of samples taken, has the effect, as Cyan said, of guaranteeing the results that the authors wanted. Note that, more generally, the fact that the authors chose to phrase their analysis so that accepting the null hypothesis was the result they wanted plus choosing a nonparametric statistical test, which is always weaker than a parametric one is in and of itself suspicious. If they had had enough samples so that it would be theoretically possible for the null hypothesis to be rejected (say if they had taken more samples) but they had still wanted the null result and they had still chosen a nonparametric test I would still be suspicious. As Cyan said, the nonparametric tests throw away most of the information.
It’s not the fault of the method if someone abuses it.
In general, no. However, if a method is more easily abused than others, that that is something worth pointing out.