To be honest I had much higher hopes for this competition than what I’ve seen.
Also seriously, top 2 entries make recommendations (lithium, paleo) with no plausible transmission mechanism. I know this is rampant is research, since you need to publish something, but I’d have hoped we’d knew better than that.
By “transmission mechanism”, do you mean “mechanism of action”? If so, when you say “with no plausible transmission mechanism”, do you mean “whose mechanism of action has not yet been discovered” or “I believe it is implausible that a mechanism of action for this could exist”?
’Cause if you mean the former, the fact that we don’t understand it shouldn’t be a barrier to using it (you might be surprised how many medications we have no idea how they work), and if you mean the latter, well, as the old saying goes, “If it happens, it’s possible”.
This is doubly true for lithium, where even if you want to reject every single one of the studies I cited, we already know it works in higher dose as a treatment for bipolar disorder.
(and I hate to follow up an interesting rationality point with a boring discussion of pharmacology, but many of the effects of lithium are known or plausibly speculated upon, including protecting neurons against glutamate excitotoxicity; I am no neuroscientist, but I know excitotoxicity has a role in dementia, probably some psychiatric disorders, and in a bunch of common neurological causes of death.)
Literally no idea how they work? Not super common, although a paper on drug targets in Nature gives this list of drugs where they can’t even begin to classify what sort of entity they act on.
But many other drugs are still incompletely understood, or only have very speculative mechanisms of actions. For example, we know that SSRI antidepressants increase levels of serotonin in the brain, but we’re not really sure why that should treat depression (the hypothesis that depression is an imbalance of neurotransmitters was originally an ad hoc hypothesis explaining why these sorts of drugs seemed to treat it, although we’ve learned a lot since then).
Paracetamol (aka acetaminophen aka Tylenol) is actually pretty mysterious and still the subject of a lot of study, as are anaesthetics (which is too bad, because if we understood how anaesthetics worked, we’d have a promising lead in figuring out what consciousness is). Most psychiatric medications range somewhere from “incompletely understood” to “might as well be witchcraft”, and lot of neurologic ones like some of the antiepileptics aren’t much better.
I was surprised to learn recently that a lot of drug discovery is now being done by brute force. For example, they discovered new cystic fibrosis drug ivacaftor by dosing lung cells in solution with more or less every known organic chemical until one of them caused the chlorine concentration of the solution to change, which indicated that it had somehow solved the error in cellular chlorine transport that causes CF.
By transmission mechanism I mean something more general. X has a non-negligible effect on Y, Y on Z etc.
An example of implausible transmission mechanism:
This or that food contains antioxidants, antioxidants protect from molecular damage, less molecular damage means you’ll live longer—it has 2 totally broken links since entirely insignificant amount of antioxidants from food get anywhere in the body, and putting more antioxidants in the cells doesn’t actually do much at all (even if reducing their amount increases damage considerably).
An example of plausible transmission mechanism:
Having dogs improve people’s moods, people in better mood have lower blood pressure, lower blood pressure decreases risk of major cardiovascular disease—we don’t have much hard data here (funnily enough they did a randomized study once, and found such effects), but every link in the chain is plausible and effect is within realistic order of magnitude.
With sufficiently overwhelming evidence it might be reasonable to ignore lack of any plausible transmission mechanism, but evidence is anything but, and I’m more inclined to think that it went from “I need to publish X papers a year” to “finshing for statistical correlations involving lithium” to “publishing a paper about that”.
A lot of people were very put off by the state of knowledge in this area; it turned out we’d chosen an area where it’s very difficult to do good work. Kevin’s entry illustrates this more than anything—he started out thinking he knew things about how to supplement, and then decided he knew far less than he thought.
Making sense of knowledge in a bad state though is precisely the sort of thing that should test rationality skills of the sort we try to cultivate here.
I think rationality makes a big difference when an area is confused, not when it’s a complex area with little data (like weather prediction without lots of sensors and supercomputers). My impression from the discussion is that supplementation is closer to the later, and not much is achieved by pure paper research.
There’s lots about lying with statistics, especially in the context of medicine. Anything by Ioannidis for one.
Arguments in general should be part of our skill set though, and those are obviously relevant here.
This confirms a prediction that I’m pretty sure I went around saying in advance (the state of knowledge in nutrition is fucked up beyond most easy epistemic wins).
Wow I’d call that confirmation bias. We got some significant solidification of existing info, a bit of new info, and spent $7500. Multiply these results by 1000 and what do you get? I’d guess, a fairly thorough knowledge-base.
When you use Word’s footnote manager, it doesn’t store the citation numbers as plain text, so if you copy paste them, it doesn’t work.
I had sent PM a slightly revised version of the paper to post for public consumption as a .docx and not the text version that was reviewed and I’ll ask them to try putting it up again.
To be honest I had much higher hopes for this competition than what I’ve seen.
Also seriously, top 2 entries make recommendations (lithium, paleo) with no plausible transmission mechanism. I know this is rampant is research, since you need to publish something, but I’d have hoped we’d knew better than that.
By “transmission mechanism”, do you mean “mechanism of action”? If so, when you say “with no plausible transmission mechanism”, do you mean “whose mechanism of action has not yet been discovered” or “I believe it is implausible that a mechanism of action for this could exist”?
’Cause if you mean the former, the fact that we don’t understand it shouldn’t be a barrier to using it (you might be surprised how many medications we have no idea how they work), and if you mean the latter, well, as the old saying goes, “If it happens, it’s possible”.
This is doubly true for lithium, where even if you want to reject every single one of the studies I cited, we already know it works in higher dose as a treatment for bipolar disorder.
(and I hate to follow up an interesting rationality point with a boring discussion of pharmacology, but many of the effects of lithium are known or plausibly speculated upon, including protecting neurons against glutamate excitotoxicity; I am no neuroscientist, but I know excitotoxicity has a role in dementia, probably some psychiatric disorders, and in a bunch of common neurological causes of death.)
How common are such medicines, in your experience?
Literally no idea how they work? Not super common, although a paper on drug targets in Nature gives this list of drugs where they can’t even begin to classify what sort of entity they act on.
But many other drugs are still incompletely understood, or only have very speculative mechanisms of actions. For example, we know that SSRI antidepressants increase levels of serotonin in the brain, but we’re not really sure why that should treat depression (the hypothesis that depression is an imbalance of neurotransmitters was originally an ad hoc hypothesis explaining why these sorts of drugs seemed to treat it, although we’ve learned a lot since then).
Paracetamol (aka acetaminophen aka Tylenol) is actually pretty mysterious and still the subject of a lot of study, as are anaesthetics (which is too bad, because if we understood how anaesthetics worked, we’d have a promising lead in figuring out what consciousness is). Most psychiatric medications range somewhere from “incompletely understood” to “might as well be witchcraft”, and lot of neurologic ones like some of the antiepileptics aren’t much better.
I was surprised to learn recently that a lot of drug discovery is now being done by brute force. For example, they discovered new cystic fibrosis drug ivacaftor by dosing lung cells in solution with more or less every known organic chemical until one of them caused the chlorine concentration of the solution to change, which indicated that it had somehow solved the error in cellular chlorine transport that causes CF.
By transmission mechanism I mean something more general. X has a non-negligible effect on Y, Y on Z etc.
An example of implausible transmission mechanism:
This or that food contains antioxidants, antioxidants protect from molecular damage, less molecular damage means you’ll live longer—it has 2 totally broken links since entirely insignificant amount of antioxidants from food get anywhere in the body, and putting more antioxidants in the cells doesn’t actually do much at all (even if reducing their amount increases damage considerably).
An example of plausible transmission mechanism:
Having dogs improve people’s moods, people in better mood have lower blood pressure, lower blood pressure decreases risk of major cardiovascular disease—we don’t have much hard data here (funnily enough they did a randomized study once, and found such effects), but every link in the chain is plausible and effect is within realistic order of magnitude.
With sufficiently overwhelming evidence it might be reasonable to ignore lack of any plausible transmission mechanism, but evidence is anything but, and I’m more inclined to think that it went from “I need to publish X papers a year” to “finshing for statistical correlations involving lithium” to “publishing a paper about that”.
A lot of people were very put off by the state of knowledge in this area; it turned out we’d chosen an area where it’s very difficult to do good work. Kevin’s entry illustrates this more than anything—he started out thinking he knew things about how to supplement, and then decided he knew far less than he thought.
Making sense of knowledge in a bad state though is precisely the sort of thing that should test rationality skills of the sort we try to cultivate here.
I think rationality makes a big difference when an area is confused, not when it’s a complex area with little data (like weather prediction without lots of sensors and supercomputers). My impression from the discussion is that supplementation is closer to the later, and not much is achieved by pure paper research.
Mathsemantics is the only book I know of on the subject. Any other suggestions?
Actually, How to Lie with Statistics (1954) might also count, but has there been any more recent work on the subject?
There’s lots about lying with statistics, especially in the context of medicine. Anything by Ioannidis for one. Arguments in general should be part of our skill set though, and those are obviously relevant here.
This confirms a prediction that I’m pretty sure I went around saying in advance (the state of knowledge in nutrition is fucked up beyond most easy epistemic wins).
Wow I’d call that confirmation bias. We got some significant solidification of existing info, a bit of new info, and spent $7500. Multiply these results by 1000 and what do you get? I’d guess, a fairly thorough knowledge-base.
Transmission mechanism? I think my paper is pretty persuasive about how dietary changes can increase mineral absorption.
By the way, do you know what happened to your footnotes? They show up like
When you use Word’s footnote manager, it doesn’t store the citation numbers as plain text, so if you copy paste them, it doesn’t work.
I had sent PM a slightly revised version of the paper to post for public consumption as a .docx and not the text version that was reviewed and I’ll ask them to try putting it up again.
Did they require .docx? How strange.