Those results look pretty weak to me, especially in the first section; with the methodology they used, I’d have expected to see something much stronger than ~35% less theism among ASD-identified individuals if there was anything to the hypothesis (owing to amplification by evaporative cooling and other group effects). With that in mind I think I find myself less confident of ASD-linked differences in religiosity than I’d have been if you asked me before reading the article, although it’s intuitively reasonable. The own-construction results do seem a little more solid, although if I’m reading this right the absolute numbers would still be low enough to admit some sampling bias.
I’m also somewhat skeptical that you could get reliable statistics on religiosity out of a pool of ~200 posts per group, especially if you’re comparing minorities in both groups; the article claims to be comparing clear self-assignments of religious belief, but I wouldn’t expect those to be all that common even on a religious discussion forum. I’d really like to know more about exactly what their raters were trained to base their estimates on and how they managed to achieve 93% reliability in a pool that small.
The design of the second section seems somewhat better, accounting for the flaws of self-assignment, but comparing self-selected Internet people to undergrads at an unspecified northeastern university still seems questionable.
Even if the results were much stronger I don’t think I’d update by much; I have a lot of concerns with their methodology here in terms of sampling and control. As you suggest a much larger sample would be an improvement but still subject to the biases that concern me.
Those results look pretty weak to me, especially in the first section; with the methodology they used, I’d have expected to see something much stronger than ~35% less theism among ASD-identified individuals if there was anything to the hypothesis (owing to amplification by evaporative cooling and other group effects). With that in mind I think I find myself less confident of ASD-linked differences in religiosity than I’d have been if you asked me before reading the article, although it’s intuitively reasonable. The own-construction results do seem a little more solid, although if I’m reading this right the absolute numbers would still be low enough to admit some sampling bias.
I’m also somewhat skeptical that you could get reliable statistics on religiosity out of a pool of ~200 posts per group, especially if you’re comparing minorities in both groups; the article claims to be comparing clear self-assignments of religious belief, but I wouldn’t expect those to be all that common even on a religious discussion forum. I’d really like to know more about exactly what their raters were trained to base their estimates on and how they managed to achieve 93% reliability in a pool that small.
The design of the second section seems somewhat better, accounting for the flaws of self-assignment, but comparing self-selected Internet people to undergrads at an unspecified northeastern university still seems questionable.
Even if the results were much stronger I don’t think I’d update by much; I have a lot of concerns with their methodology here in terms of sampling and control. As you suggest a much larger sample would be an improvement but still subject to the biases that concern me.