I’m afraid it’s critical. They claim to have shown that their treatment allows rats to remember objects for up to 24 weeks after seeing them, while normal rats could remember objects for only 45 minutes. They did a long series of tests with the treated rats to demonstrate this. But they appear to have stopped testing the normal rats after they “failed” the 60-minute test.
Similarly, they tested the treated rats for memory of up to 6 objects. They stopped testing the normal rats after they “failed” the test for 4 objects (figure 1D) - although, again, they spent no more time examining the old objects in that test; they merely spent less time examining new objects.
If the treatment had any effect, it appears to me that it affected the rats’ curiousity, not their memory. But the most likely explanation is that the normal rats failed those 2 critical tests purely by chance, perhaps because something startled them (e.g., there was a hawk overhead during the exposure to the new objects).
The sample size is 16, which should be enough. A random factor shouldn’t have that big an effect if the trials were uncorrelated. To make the trials uncorrelated, they would need to interleave their trials. For instance, if they do all the 30-minute tests, then all the 45-minute tests, then all the 60-minute tests, each group of tests is almost completely correlated in environmental conditions. Because rats have such different senses than humans, it’s impossible for a human to tell by observation whether something unusual to a rat is going on.
The expectation of the experimenter is another factor that can correlate results within a trial group. We usually require double-blind tests on humans, but not on rats.
I’m afraid it’s critical. They claim to have shown that their treatment allows rats to remember objects for up to 24 weeks after seeing them, while normal rats could remember objects for only 45 minutes. They did a long series of tests with the treated rats to demonstrate this. But they appear to have stopped testing the normal rats after they “failed” the 60-minute test.
Similarly, they tested the treated rats for memory of up to 6 objects. They stopped testing the normal rats after they “failed” the test for 4 objects (figure 1D) - although, again, they spent no more time examining the old objects in that test; they merely spent less time examining new objects.
If the treatment had any effect, it appears to me that it affected the rats’ curiousity, not their memory. But the most likely explanation is that the normal rats failed those 2 critical tests purely by chance, perhaps because something startled them (e.g., there was a hawk overhead during the exposure to the new objects).
Back up—are you suggesting that a random factor could have that big an effect on the results? How small are their sample sizes?
The sample size is 16, which should be enough. A random factor shouldn’t have that big an effect if the trials were uncorrelated. To make the trials uncorrelated, they would need to interleave their trials. For instance, if they do all the 30-minute tests, then all the 45-minute tests, then all the 60-minute tests, each group of tests is almost completely correlated in environmental conditions. Because rats have such different senses than humans, it’s impossible for a human to tell by observation whether something unusual to a rat is going on.
The expectation of the experimenter is another factor that can correlate results within a trial group. We usually require double-blind tests on humans, but not on rats.