Er… how incompetent do you think these researchers are?
You need to control for intelligence, all you have is IQ test which is intelligence measured with some errors (edit: to be precise, the correlation between IQ test score and “general intelligence” is presumed to be around 0.8 or less), do I need to spell it out for you that controlling for something measured with an error does not result in perfect controls? The level of competence in psychology is pretty low.
tl;dr; of course they didn’t control for intelligence. They controlled for IQ, which is something correlated with intelligence (but not very well). You can pick people, measure the math portion of IQ test, then ‘control for intelligence’ meaning the rest of the IQ test, you still have the people with better mathematical intelligence being more correct about stuff including non-existence of god edit: and still do better on some other IQ test that doesn’t correlate perfectly with the first one.
edit: just look at the “cognitive reflection test”:
A bat and a ball cost $1.10 in total. The bat costs $1.00 more than the ball.
How much does the ball cost?
If it takes 5 machines 5 minutes to make 5 widgets, how long would it take
100 machines to make 100 widgets?
In a lake, there is a patch of lily pads. Every day, the patch doubles in size.
If it takes 48 days for the patch to cover the entire lake, how long would it
take for the patch to cover half of the lake?
the first one, i do admit at the 0.1 popping up, but obviously—having received math education—i habitually test answers and it takes fraction of a second—note that testing your answers doesn’t imply you aren’t answering intuitively and note that you can’t test ‘does god exist’ in similar fashion. The remaining two, I have to stop and think to come up with allegedly ‘intuitive’ answers. My suspicion is that they pop up because of some childhood conditioning that if you see numbers you must do some math operation on them.
You need to control for intelligence, all you have is IQ test which is intelligence measured with some errors (edit: to be precise, the correlation between IQ test score and “general intelligence” is presumed to be around 0.8 or less), do I need to spell it out for you that controlling for something measured with an error does not result in perfect controls? The level of competence in psychology is pretty low.
This is a fully general counter-argument. You can dismiss any test in psychology whatsoever by claiming it does not track the underlying property to your arbitrary demands. (What, 0.8 is not enough?)
Some causal diagrams and numerical calculations clarify the argument. Call IR the score on the supposed test for intuitive vs. reflective style (for me the right answers are all intuitive), BG the score for belief in god, IQ the score on an IQ test, and G the hypothetical true intelligence. Consider the causal diagram that has an arrow from G to each of IQ, IR, and BG, and no other arrows. This is the causal diagram that controlling for intelligence is intended to rule out by finding a conditional correlation that is incompatible with it.
To really control for intelligence would require matching subjects for G value. This cannot be done, so IQ is used as a proxy, but IQ correlates only 0.8 with G. The question is then: when you control for IQ, how well are you controlling for G? How much correlation due to that possible causal diagram are you squeezing out of the relationship between IR and BG? I’ve only glanced at the paper and do not know if the authors have considered this question.
I shall suppose for the sake of example, and because this example is one that I happen to have worked out the numbers for, that all the distributions involved are multivariate Gaussian. IQ and G have a product-moment correlation of c = 0.8. By what factor does knowing someone’s IQ narrow the distribution of their G? The answer is 1/sqrt(1-c^2) = 1.667.
That doesn’t look like very much, does it? Is it enough to rule out the correlation between IR and BG as being due solely to a dependence of each on G? Well, what was the unconditional correlation between IR and BG? The paper is all F-values and NHST, but the impression I get from their Figure 1 is around 0.5 to 0.8.
So, consider a hypothetical model in which IR = G + noiseIR, BG = G + noiseBG, and IQ = G + noiseIQ, the three noises being independent. This is the model that we are trying to rule out by controlling for IQ. Choose units so that all variables have mean 0, and G has variance 1. Clearly, the conditional correlation of IR and BG given G is zero, and the experiment would find against that. But how high could the conditional correlation of IR and BG be, when what is fixed is not G but IQ?
We have IR = IQ—noiseIQ + noiseIR and BG = IQ—noiseIQ + noiseBG, so their conditional correlation given IQ is the unconditional correlation between noiseIR-noiseIQ and noiseBG-noiseIQ. Calling the variances of these noises vnIQ, vnIR, and vnBG, the correlation is 1/sqrt((1 + vnIR/vIQ)(1 + vBG/vIQ)).
Choosing our units earlier means that all of the variances vIQ, vIR, and vBG are relative to the variance of G. This tells us the value of vIQ: a correlation of c=0.8 between G + noiseIQ and G implies that vIQ = 1/c^2 − 1 = 9⁄16. We don’t know vIR or vBG, but the correlation between G + noiseIR and G + noiseBG is 1/sqrt((1 + vIR)(1 + vnBG)), and we estimated this to be around 0.5 to 0.8.
Given the ratio of vnIR to vnBG (which we do not know) this determines the correlation between IR and BG conditional on IQ. My calculations indicate that the correlation is very insensitive to the ratio vnIR/vnBG over a range of 1⁄4 to 4, so I’ll just give results where vnIR=vnBG.
For an unconditional correlation cIRBG between IR and BG ranging between 0.5 and 0.8, here are the correlations cIRBGIQ conditional on IQ:
You get it wrong. It’s their responsibility to show 0.8 is enough. I dunno if it works as a general counter argument in psychology, but in physics (and non-diseased disciplines in general) you do have to account for errors in your measurement apparatus as a possible source for your correlations, and if you don’t, no-one is obligated to believe you.
edit: imagine you had been measuring speed via proxy that has at best 0.8 correlation with speed.
edit: okay, the article seem to find stronger correlation between belief in god and the failures at their test, than between belief in god, and IQ. At same time, there’s many ways to describe what their test measure, e.g. intellectual laziness would do as well (the fallacious answer is a mathematical operation on last 2 numbers you heard—won’t get simpler than this), or the arrogance (never self doubting enough to check if the answer makes sense).
You need to control for intelligence, all you have is IQ test which is intelligence measured with some errors (edit: to be precise, the correlation between IQ test score and “general intelligence” is presumed to be around 0.8 or less), do I need to spell it out for you that controlling for something measured with an error does not result in perfect controls? The level of competence in psychology is pretty low.
tl;dr; of course they didn’t control for intelligence. They controlled for IQ, which is something correlated with intelligence (but not very well). You can pick people, measure the math portion of IQ test, then ‘control for intelligence’ meaning the rest of the IQ test, you still have the people with better mathematical intelligence being more correct about stuff including non-existence of god edit: and still do better on some other IQ test that doesn’t correlate perfectly with the first one.
edit: just look at the “cognitive reflection test”:
the first one, i do admit at the 0.1 popping up, but obviously—having received math education—i habitually test answers and it takes fraction of a second—note that testing your answers doesn’t imply you aren’t answering intuitively and note that you can’t test ‘does god exist’ in similar fashion. The remaining two, I have to stop and think to come up with allegedly ‘intuitive’ answers. My suspicion is that they pop up because of some childhood conditioning that if you see numbers you must do some math operation on them.
This is a fully general counter-argument. You can dismiss any test in psychology whatsoever by claiming it does not track the underlying property to your arbitrary demands. (What, 0.8 is not enough?)
Good day.
Some causal diagrams and numerical calculations clarify the argument. Call IR the score on the supposed test for intuitive vs. reflective style (for me the right answers are all intuitive), BG the score for belief in god, IQ the score on an IQ test, and G the hypothetical true intelligence. Consider the causal diagram that has an arrow from G to each of IQ, IR, and BG, and no other arrows. This is the causal diagram that controlling for intelligence is intended to rule out by finding a conditional correlation that is incompatible with it.
To really control for intelligence would require matching subjects for G value. This cannot be done, so IQ is used as a proxy, but IQ correlates only 0.8 with G. The question is then: when you control for IQ, how well are you controlling for G? How much correlation due to that possible causal diagram are you squeezing out of the relationship between IR and BG? I’ve only glanced at the paper and do not know if the authors have considered this question.
I shall suppose for the sake of example, and because this example is one that I happen to have worked out the numbers for, that all the distributions involved are multivariate Gaussian. IQ and G have a product-moment correlation of c = 0.8. By what factor does knowing someone’s IQ narrow the distribution of their G? The answer is 1/sqrt(1-c^2) = 1.667.
That doesn’t look like very much, does it? Is it enough to rule out the correlation between IR and BG as being due solely to a dependence of each on G? Well, what was the unconditional correlation between IR and BG? The paper is all F-values and NHST, but the impression I get from their Figure 1 is around 0.5 to 0.8.
So, consider a hypothetical model in which IR = G + noiseIR, BG = G + noiseBG, and IQ = G + noiseIQ, the three noises being independent. This is the model that we are trying to rule out by controlling for IQ. Choose units so that all variables have mean 0, and G has variance 1. Clearly, the conditional correlation of IR and BG given G is zero, and the experiment would find against that. But how high could the conditional correlation of IR and BG be, when what is fixed is not G but IQ?
We have IR = IQ—noiseIQ + noiseIR and BG = IQ—noiseIQ + noiseBG, so their conditional correlation given IQ is the unconditional correlation between noiseIR-noiseIQ and noiseBG-noiseIQ. Calling the variances of these noises vnIQ, vnIR, and vnBG, the correlation is 1/sqrt((1 + vnIR/vIQ)(1 + vBG/vIQ)).
Choosing our units earlier means that all of the variances vIQ, vIR, and vBG are relative to the variance of G. This tells us the value of vIQ: a correlation of c=0.8 between G + noiseIQ and G implies that vIQ = 1/c^2 − 1 = 9⁄16. We don’t know vIR or vBG, but the correlation between G + noiseIR and G + noiseBG is 1/sqrt((1 + vIR)(1 + vnBG)), and we estimated this to be around 0.5 to 0.8.
Given the ratio of vnIR to vnBG (which we do not know) this determines the correlation between IR and BG conditional on IQ. My calculations indicate that the correlation is very insensitive to the ratio vnIR/vnBG over a range of 1⁄4 to 4, so I’ll just give results where vnIR=vnBG.
For an unconditional correlation cIRBG between IR and BG ranging between 0.5 and 0.8, here are the correlations cIRBGIQ conditional on IQ:
You would need a much higher correlation betweeen IQ and G to push cIRBGIQ close to zero:
You get it wrong. It’s their responsibility to show 0.8 is enough. I dunno if it works as a general counter argument in psychology, but in physics (and non-diseased disciplines in general) you do have to account for errors in your measurement apparatus as a possible source for your correlations, and if you don’t, no-one is obligated to believe you.
edit: imagine you had been measuring speed via proxy that has at best 0.8 correlation with speed.
edit: okay, the article seem to find stronger correlation between belief in god and the failures at their test, than between belief in god, and IQ. At same time, there’s many ways to describe what their test measure, e.g. intellectual laziness would do as well (the fallacious answer is a mathematical operation on last 2 numbers you heard—won’t get simpler than this), or the arrogance (never self doubting enough to check if the answer makes sense).