Instead, you select from a population which is as similar as possible to the treatment group
They did this with an earlier batch (I was part of that control group) and they haven’t reported that data. I found this disappointing, and it makes me trust this round of data less.
On Sunday, Sep 8, 2013 Dan at CFAR wrote:
Last year, you took part in the first round of the Center for Applied Rationality’s study on the benefits of learning rationality skills. As we explained then, there are two stages to the survey process: first an initial set of surveys in summer/fall 2012 (an online Rationality Survey for you to fill out about yourself, and a Friend Survey for your friends to fill out about you), and then a followup set of surveys one year later in 2013 when you (and your friends) would complete the surveys again so that we could see what has changed.
You’re right, we should’ve posted the results on our previous study. I’ll put those numbers together in a comprehensible format and then I’ll have them posted soon.
The brief explanation of why we didn’t take the time to write them up earlier is that the study was underpowered and we thought that the results weren’t that informative. In retrospect, that decision was a mistake.
I’ve put a list of the workshop surveys that we’ve done in a separate comment.
They did this with an earlier batch (I was part of that control group) and they haven’t reported that data. I found this disappointing, and it makes me trust this round of data less.
You’re right, we should’ve posted the results on our previous study. I’ll put those numbers together in a comprehensible format and then I’ll have them posted soon.
The brief explanation of why we didn’t take the time to write them up earlier is that the study was underpowered and we thought that the results weren’t that informative. In retrospect, that decision was a mistake.
I’ve put a list of the workshop surveys that we’ve done in a separate comment.