That is a really good point, and it helps me think much more clearly about the comparison I made.
You’re completely right about the lack of a control group. If we assume that tracking any similar cohort (matched for age, gender, education, and an active interest in self-improvement) would naturally show similar upward trends over a year, then attributing the effect entirely to the retreat is definitely flawed.
When I wrote that, I was operating under the baseline assumption that without a specific treatment or change in circumstance, life satisfaction and those other metrics would remain relatively static. However, I haven’t actually read the literature on the natural trajectory for this specific demographic—it’s entirely possible that a similar cohort would see an organic increase anyway simply because they are highly motivated people looking to improve. I don’t have enough confidence to make a strong claim either way.
A much better study design would use an active control group, for example, comparing CFAR attendees to people attending Less.online meetups or similar events. That would help us figure out if the measured effects actually come from the CFAR curriculum, or if they just stem from the baseline benefits of aggregating rationalists together and having a fun gathering.
That is a really good point, and it helps me think much more clearly about the comparison I made.
You’re completely right about the lack of a control group. If we assume that tracking any similar cohort (matched for age, gender, education, and an active interest in self-improvement) would naturally show similar upward trends over a year, then attributing the effect entirely to the retreat is definitely flawed.
When I wrote that, I was operating under the baseline assumption that without a specific treatment or change in circumstance, life satisfaction and those other metrics would remain relatively static. However, I haven’t actually read the literature on the natural trajectory for this specific demographic—it’s entirely possible that a similar cohort would see an organic increase anyway simply because they are highly motivated people looking to improve. I don’t have enough confidence to make a strong claim either way.
A much better study design would use an active control group, for example, comparing CFAR attendees to people attending Less.online meetups or similar events. That would help us figure out if the measured effects actually come from the CFAR curriculum, or if they just stem from the baseline benefits of aggregating rationalists together and having a fun gathering.