Steve here is a note I sent to the FDA a week or two ago.
“I am writing to comment on the meeting to be held March 8-9 about direct to consumer (DTC) genetic testing (Docket FDA-2011-N-0066). I am especially motivated to write after reading the plea to you by the AMA that any DTC results of possible medical interest be censored to consumers. Their letter reflects an appalling paternalistic arrogance that would violate basic freedoms and impede public scientific understanding. I presume that if they could they would have you ban bathroom scales on the grounds that body weight must only be revealed in consultation with a “qualified medical professional.”
The AMA submission has two main themes. The first is that citizens are unable to understand the risks and predicted outcomes that might be reported and that experts are vital to provide guidance. My own experience is that I am perfectly capable of finding empirical risks from current literature, I expect I can do a much better and more thorough job than my personal physician, and even my teenage son can do it with no trouble. My own experience, again, is that only about 1 in 5 medical students know what Bayes’ Theorem is.
The second theme is that knowledge of potentially medically relevant genotypes can do some unspecified harm to customers. I have spent a total of six or so years on university IRBs, and this kind of worry is ever present. While there is much public loose talk about psychological harm and the like, within the committee room we all understand that the practice of withholding any data from subjects about themselves is nothing but protection from lawyers. I am perfectly free to refuse to participate in research and in clinical trials but I am not free to refuse to participate in federal censorship of knowledge of my own genotype.
I would urge you to keep freedom of information for consumers at the center of the table when you discuss regulation of the DTC genetic testing industry.”
I’m surprised there isn’t more activity about this here considering the strong presence of transhumanists.
My own experience, again, is that only about 1 in 5 medical students know what Bayes’ Theorem is.
Just like how 1 in 5 medical students knows what the Hindu-Arabic number system is. So what? The whole problem is that medical students excel at rote memorization at the expense of genuine, model-based understanding.
What matters is whether med students can perform the kind of statistical inference codified by Bayes’s Theorem—whatever they call it—not whether they can spit out the right password in response to the question, “What is Bayes’s Theorem?” (and then go on to unjustifiably tell their patients they have breast cancer).
Just like how 1 in 5 medical students knows what the Hindu-Arabic number system is.
Over here the Arabic and Roman numeral systems are compared and contrasted in primary school with a short word on both their development. This is done again with a bit more history in High School. I’m quite confident that if you said “Arabic numerals”, 90%+ of Medical students would know what you are talking about. And in context 80%+ would infer or remember what the Hindu-Arabic numeral system is.
But me pointing out that your own password guessing example is country dependent or perhaps even false is utterly irrelevant because it makes a vivid example.
Its highly probable that Henry Harpending has a good handle on the difference between password guessing and knowledge it is reasonable to assume he meant that less than 1 in 5. can perform the kind of statistical inference codified by Bayes’s theorem. Which I think you would agree with considering you, in my opinion accurately, stated that :
The whole problem is that medical students excel at rote memorization at the expense of genuine, model-based understanding.
Remember the sentence you are criticizing was a quote from a note he sent to the FDA not a quote from a rationalist discussion. Saying that:
that only about 1 in 5 medical students know what Bayes’ Theorem is.
Is much more expedient than explaining what one really means. Short notes are more likley to get read. It also has the advantage of sounding bad to someone who has no inkling of what Bayes’ Theorem or the knowledge that password is associated with is.
When one is sending a letter aimed at convincing someone of your point it is optimized only to convince.
So if I understand you correctly, HH’s phrasing of his complaint about an irrelevant deficiency (whether the students know “Bayes’s Theorem”) is understandable because there’s no similarly-expedient way to convey the same problem. I disagree. Watch:
“only about 1 in 5 medical students … … can perform Bayesian inference.” [1] … understand the impact of false positives in medical tests.” … reason consistently with Bayes’s Theorem.” … can correctly estimate false positive rates.”
I don’t think those rephrasings require any kind of brilliance, just a willingness to convey what you actually mean, and looking for a way to do so.
When one is sending a letter aimed at convincing someone of your point it is optimized only to convince.
I hope not, because that’s all the more reason for someone to ignore it! (If someone is willing to say anything to make a point, the listener should not expect a helpful relationship between the arguments and reality.) It should be optimized for persuasiveness under the constraint that one only say stuff that is actually relevant and true. No such constraint → no reason to care what they have to say.
If it seems like I’m belaboring a minor point, it’s because HH’s phrasing implies that the important thing is for students to know this or that password—the exact mentality we want to get rid of.
[1] Note: there is a huge difference between whether one can “perform” Bayesian inference—which happened long before Bayes walked the earth—and whether someone “knows what’s Bayes’s Theorem is”.
I hope not, because that’s all the more reason for someone to ignore it! (If someone is willing to say anything to make a point, the listener should not expect a helpful relationship between the arguments and reality.) It should be optimized for persuasiveness under the constraint that one only say stuff that is actually relevant and true. No such constraint → no reason to care what they have to say.
I think this goes without saying and all falls under optimizing to convince someone.
Its a queer scenario where you could get away with with not saying anything relevant and actually have this be optimal. But saying that the most persuasive argument possible is one where everything said is relevant and true is a bit hopelessly naive.
What really made your point is this:
So if I understand you correctly, HH’s phrasing of his complaint about an irrelevant deficiency (whether the students know “Bayes’s Theorem”) is understandable because there’s no similarly-expedient way to convey the same problem. I disagree.
And the examples you give.
In any case on second thought Harpending probably wasn’t optimizing only for persuasiveness (though one might argue he should). I think he most likley was pretty cost efficient with regards to the amount of thought and effort he put into the note and its expected effect. So while I now concede your point I don’t think this is particularly damning criticism. LW standards on this are, relatively speaking, pretty high.
Comment on isteve by HH:
I’m surprised there isn’t more activity about this here considering the strong presence of transhumanists.
Just like how 1 in 5 medical students knows what the Hindu-Arabic number system is. So what? The whole problem is that medical students excel at rote memorization at the expense of genuine, model-based understanding.
What matters is whether med students can perform the kind of statistical inference codified by Bayes’s Theorem—whatever they call it—not whether they can spit out the right password in response to the question, “What is Bayes’s Theorem?” (and then go on to unjustifiably tell their patients they have breast cancer).
Over here the Arabic and Roman numeral systems are compared and contrasted in primary school with a short word on both their development. This is done again with a bit more history in High School. I’m quite confident that if you said “Arabic numerals”, 90%+ of Medical students would know what you are talking about. And in context 80%+ would infer or remember what the Hindu-Arabic numeral system is.
But me pointing out that your own password guessing example is country dependent or perhaps even false is utterly irrelevant because it makes a vivid example.
Its highly probable that Henry Harpending has a good handle on the difference between password guessing and knowledge it is reasonable to assume he meant that less than 1 in 5. can perform the kind of statistical inference codified by Bayes’s theorem. Which I think you would agree with considering you, in my opinion accurately, stated that :
Remember the sentence you are criticizing was a quote from a note he sent to the FDA not a quote from a rationalist discussion. Saying that:
Is much more expedient than explaining what one really means. Short notes are more likley to get read. It also has the advantage of sounding bad to someone who has no inkling of what Bayes’ Theorem or the knowledge that password is associated with is.
When one is sending a letter aimed at convincing someone of your point it is optimized only to convince.
So if I understand you correctly, HH’s phrasing of his complaint about an irrelevant deficiency (whether the students know “Bayes’s Theorem”) is understandable because there’s no similarly-expedient way to convey the same problem. I disagree. Watch:
“only about 1 in 5 medical students …
… can perform Bayesian inference.” [1]
… understand the impact of false positives in medical tests.”
… reason consistently with Bayes’s Theorem.”
… can correctly estimate false positive rates.”
I don’t think those rephrasings require any kind of brilliance, just a willingness to convey what you actually mean, and looking for a way to do so.
I hope not, because that’s all the more reason for someone to ignore it! (If someone is willing to say anything to make a point, the listener should not expect a helpful relationship between the arguments and reality.) It should be optimized for persuasiveness under the constraint that one only say stuff that is actually relevant and true. No such constraint → no reason to care what they have to say.
If it seems like I’m belaboring a minor point, it’s because HH’s phrasing implies that the important thing is for students to know this or that password—the exact mentality we want to get rid of.
[1] Note: there is a huge difference between whether one can “perform” Bayesian inference—which happened long before Bayes walked the earth—and whether someone “knows what’s Bayes’s Theorem is”.
I think this goes without saying and all falls under optimizing to convince someone.
Its a queer scenario where you could get away with with not saying anything relevant and actually have this be optimal. But saying that the most persuasive argument possible is one where everything said is relevant and true is a bit hopelessly naive.
What really made your point is this:
And the examples you give.
In any case on second thought Harpending probably wasn’t optimizing only for persuasiveness (though one might argue he should). I think he most likley was pretty cost efficient with regards to the amount of thought and effort he put into the note and its expected effect. So while I now concede your point I don’t think this is particularly damning criticism. LW standards on this are, relatively speaking, pretty high.