their only RCT analysis finds the worst results for napping on alertness, with the central estimate showing a negative impact
What you mean is that it showed a difference between the napping and control groups at t1 that favored the control group. This seems to have been due to baseline differences between the groups, because when you look at how the mean alertness of each group changed in the RCT analysis, they both improved, with that of the nap group having improved a bit more:
That’s a good catch. So it sounds like they’re comparing the “effects of a nap” by comparing the control group performance at t1 with the nap group performance at t1, rather than the difference in the magnitude of improvement between the groups from time t0 to t1?
Example:
Controls score 1 for alertness at time t0, and a 3 at time t1.
Nappers score a −1 for alertness at time t0, and a 2 at time t1.
The nappers improved by 3, while the controls only improved by 2.
Yet the controls scored higher at t1, because they started at higher baseline alertness.
If this is what’s going on, my credence in napping being effective is increased. The gains in figure S4 vs. S7 are easy to discern, just by eye.
So it sounds like they’re comparing the “effects of a nap” by comparing the control group performance at t1 with the nap group performance at t1, rather than the difference in the magnitude of improvement between the groups from time t0 to t1?
Yeah, that is what Figure 4 (which you attached in your comment) shows, and what the effect sizes reported in the abstract refer to. There does seem to be a trend towards nap groups across the studies having a bit lower baseline cognitive performance:
Figure S6 directly compares the change in performance of the groups, though the effect sizes are harder to interpret in it.
What you mean is that it showed a difference between the napping and control groups at t1 that favored the control group. This seems to have been due to baseline differences between the groups, because when you look at how the mean alertness of each group changed in the RCT analysis, they both improved, with that of the nap group having improved a bit more:
That’s a good catch. So it sounds like they’re comparing the “effects of a nap” by comparing the control group performance at t1 with the nap group performance at t1, rather than the difference in the magnitude of improvement between the groups from time t0 to t1?
Example:
Controls score 1 for alertness at time t0, and a 3 at time t1.
Nappers score a −1 for alertness at time t0, and a 2 at time t1.
The nappers improved by 3, while the controls only improved by 2.
Yet the controls scored higher at t1, because they started at higher baseline alertness.
If this is what’s going on, my credence in napping being effective is increased. The gains in figure S4 vs. S7 are easy to discern, just by eye.
Yeah, that is what Figure 4 (which you attached in your comment) shows, and what the effect sizes reported in the abstract refer to. There does seem to be a trend towards nap groups across the studies having a bit lower baseline cognitive performance:
Figure S6 directly compares the change in performance of the groups, though the effect sizes are harder to interpret in it.
Thanks for digging out the supplemental figures! Interesting...