Alternative Places to Get Ideas (Also, “In Defense of Food”)
I discovered Less Wrong a few months ago (courtesy of Harry Potter and the Methods of Rationality), and I am extremely grateful to have such a thoughtful place to have discussions and to learn new things. But one of the more significant hazards I’ve become aware of is confirmation bias. And since I began coming here a lot, I find that I evaluate new information through a “Less Wrong Lens” which officially means “well researched and thought out” which is perfectly fine but also includes some new or reinforced biases.
I recently realized the extent of the problem while reading “In Defense of Food”. The basic premise (or most relevant one) of the book is that while science may one day be able to determine what is healthy on a nutrient-by-nutrient basis and let us craft whatever artificial foods we want, we are not nearly at that point yet. Every few years the prevailing beliefs of the nutritionist and scientific communities change, people scurry to catch up, and regardless, since transition from a “traditional” to “scientific” diet, certain nutrition-related diseases have gotten more prevalent, not less.
His argument is that traditional diets have often had thousands of years to evolve to match the needs and available food sources of populations. So while the variables and interactions may be complex enough that we don’t know why, until science DOES figure it out, individual people are better off sticking to the diets of the past. At the same time, corporations that are interested only in marketing as much food as quickly as possible benefit from constantly changing scientific attitudes. (Disclaimer: yes, I’m oversimplifying again, but my point isn’t even necessarily about the merits of the book so bear with me).
By the end of the book, I did agree with his basic premises. I’m not sure his solution is the single best one, but it’s a large step up from the diet that the average westerner is going to have. It wasn’t explicitly anti-science, just attempting to be realistic about what science can realistically accomplish, and what unique pitfalls come from giving science (or more importantly, politicians and corporations on whom scientists are dependent for funding) the position of power that religion and other cultural institutions once had.
But the first half of the book does have a pretty obvious goal, not of discrediting science per se, but “taking the wind out of science’s sails”. And in the wake of reading “Methods of Rationality” it absolutely rankled me. I could feel my memetic immune system going into overdrive, looking for reasons not to believe whatever the man ended up having to say. I’m currently unsure whether that had to do with the way he was writing, if he did have his own ax to grind, or if it was purely my own biases coloring the words. But whatever his motivations, I’m grateful for having found the book at the time I did, because it taught me a lesson about my own mind. My answer isn’t to say “oh, science is just as flawed as everything else now,” but I think I’ll be able to approach things from a more neutral perspective.
Now, my purpose of posting this is two-fold. One, is I’m simply curious if other people had read the book and had anything to say about it, one way or another. But the other is to ask: do you have any sources you turn to specifically to help broaden your horizon from the prevailing mindset at Less Wrong? Websites that are not “anti-rational” or “anti-science,” that you’d still consider trustworthy sources of information, but that help to offset certain biases that you might have accumulated here?
We have to be careful when thinking about “science” as a single entity; the science that physicists do, the science that biologists, and the science that nutritionists do are each very different.
My take on what happened to nutrition science is that the nutrition science research community settled on a paradigm (controlled dietary studies followed by measuring indirect proxies for health) that was inadequate. They then put out a bunch of studies, each of which was only very weak evidence and had an extraordinarily long list of caveats. This got amplified first by reporting p-values which failed to account for those caveats, and then again by the media; and the result was a bunch of dietary recommendations that were some combination of noise, echo chamber effects, and deliberate manipulation, with barely any signal.
But that isn’t a failure of science, per se. That’s a failure of the research and publication methodologies of one particular field. It is concerning that other fields are using similar publication methodologies (especially the use of p-values), and there are some other fields where there is reason to suspect that the signal to noise ratio is also bad. The lesson I take from nutrition science is that you can’t trust a community’s output just because they call their work “science” and have all the trappings thereof; you have to look closely, see if it makes sense, and see how far above the noise floor their models’ predictions really are.
There is a decent talk “Big Fat Fiasco” (about an hour long) that explains what happened with nutrition science.
Some of the interesting parts are near the end of part 2/start of part 3:
Specifically senator McGovern dismissing complaints from scientists that there was not enough evidence that fat caused hear disease by saying:
A little latter the video mentions that at the time 90% of all funding for research on heart disease was provided by the US Government and American Heart Association. Thus once both said that fat causes heart disease it was nearly impossible for scientists who got conflicting reports to get funding.
Edited conclusion to make it clearer:
There are two problems here:
1) The attitude that this area is too important to wait for “every last shred of evidence” and thus we must go with science based upon weak evidence.
Where else have I heard this, which appears to be prevalent in climate science and pandemic medicine today.
2) Nearly all funding provided by a few large organizations that are thus subject to politics and group think.
This appears to be true in most sciences today.
As such, unfortunately, it appears the case nutrition science isn’t just an isolated incident.
I suppose the lesson here is that an inability to wait until all evidence is in to being to act does not imply that you should stop investigating new evidence once you begin to act.
The problem is that once you begin to act you’re subject to commitment bias. Namely, as happened in the example, you have a psychological and possibly institutional commitment to the correctness of the theory you’re acting under.
I think not waiting for every last shred of evidence is the root of Bayesian thinking and is also the justification for considering there will be a singularity and that there are existential risks that we can do something about now. Well before the last shred of evidence is even remotely in sight.
The last shred of evidence about existential risks is one of them killing everyone.
The other side of that coin is that we should stop treating criticisms and/or attacks on individual scientific theories as attacks on “science”.
I mostly agree but am bothered by the fact that from an outside view this sounds like No True Scotsman.
This is a valid point. However, there is an objective fact that’s different between physics/biology and nutrition: in the former, there is a lot of historical progress: stuff discovered and promoted at a high confidence tends to be supported and replicated. In the latter, stuff promoted at high confidence by the media is fairly likely to be contradicted again soon after. So it’s significantly more reasonable to ignore the results of nutrition science when deciding what to eat than it is to ignore the predictions of, say, biology when deciding whether to vaccinate your children.
I suspect much of medicine especially the newer stuff is probably nearly as bad as nutrition.
Edit: See Robin Hanson’s many posts on the subject.
Why especially the “newer stuff”?
The two most obvious reasons are:
1) Once the low-hanging fruit is exhausted, people are more likely to make stuff up.
2) Newer stuff has had less time for problems to get exposed.
Just curious, as I’ve heard the opposite asserted with confidence.
1) Very little of the Hansonian critique of medicine involves researchers making stuff up, and I doubt this is a major problem.
2) True, although hopefully research methodology is improving.
This analysis may interest you, I seem to recall it supports your suspicion.
Sorry about that, I didn’t meant more generate results based on statistical noise, then outright faking research.
Ah. Then yeah that’s a problem, but I’m not sure why this would be worse with recent research.
This article gives a pretty good overview of the shortcomings of medical statistics, and includes one of my favorite lines ever:
Because the earlier stuff, e.g., sanitation, vaccines, and antibiotics, had a stronger effect and thus was easier to notice above the noise.
And also the older stuff has been around long enough to get the bad results beaten out of it.
Oppenheimer’s maxim is relevant here:
A general theory of nutrition would be highly useful, but there’s no reason to believe modern scientific methodology can produce such a thing.
It’s not necessarily the science’s fault that nutrition is an easy place for pseudoscience to take root. One might look at immunology, or psychology, and see nearly identical situations. People seem to think that the science of nutrition is uncertain or self-contradictory, mixing the science with the headlines.
You might want to read Ben Goldacre’s book Bad Science. He discusses specific issues that have made nutrition an area that has less good science occurring in it, including people with no qualifications being labeled as nutritionists, and a general tendency to confuse correlation with causation. Some of the issues that are happening in nutrition are more or less unique to that area.
Although note that academic research here isn’t the same as the industry in general. The journals are probably less influenced by the industry than trials of pharmaceuticals.
Yes, areas related to drug research have a whole different set of problems (Goldacre discusses those as well).
I haven’t read the book although I read about it in nutrition related news sources when it was published. I’m personally a hobby reader of nutrition science since a couple of years, and what put me somewhat off regarding this book was this motto of the authors: “Eat food. Not too much. Mostly plants.”
The reason is that—without going into the details—I am confident that the “mostly plants” part is false. Skimming the reviews, looking through the table of contents and reading some pages (all through Amazon) I just subtracted several further points from the book because of numerous scientific inaccuracies, overly polemic tone and the usual vacuous blaming of food industries and nutrition science. And yep, the identification of nutritionism as a form of evil raises another red flag. It seems to me that the author justifies ad-hoc conclusions with the (basically false) argument that “foods can’t be reduced to the constituent chemicals”, and then sends several circular justifications forth and back.
The harm done by margarine, wheat, sugar etc. is the consequence of doing bad science and reaching incorrect conclusions. The remedy, however, comes from doing good science and honest reductionism, not from espousing various forms of non-science and magical thinking.
For comparison, these three blogs are pretty well-researched and overall well-written: Whole Health Source, Daily Lipid, Raw Food SOS
I haven’t read the book either but am wondering how on earth he comes up with ‘mostly plants’ given that the three harmful products just mentioned:
are all plant products.
Especially looking at margarine, which is a plant product whose substitution for animal products (butter and lard) turned out to have been a bad idea.
but they aren’t Food, at least not according to Pollan. He tries to emphasize that people used to be much healthier because they ate Things, rather than Parts of Things. He claims that historically, when we take a plant and process it to get a single nutrient out of it, we are likely to produce much less healthful food, and that applies to pretty much all three of your points, with the possible exception of wheat (especially if you eat most of the plant).
This is his argument, not mine, but it’s one I’m moderately swayed by.
By the end of the book he’s also clarified that it’s not that food reductionism doesn’t work, period. It’s just that it’s not nearly at a point yet where we actually know what we’re doing.
Right. It’s very important to note (especially around here) that he’s not anti-science, he’s just pointing out that the current state of nutrition science is awful and that we’re currently not ready to custom-design our food.
I also more-or-less agree with this. I also think that for similar reasons it makes sense to get your protein from meat rather then processed soy or whatever.
For the record, I get my protein from dairy and eggs, which is basically the same. people vastly overestimate how much protein they need. Three glasses of milk is basically it. It’s also perfectly healthy to get your protein from rice and beans and similar combinations.
But I do eat a lot of processed soy products, and the book has made me rethink that a bit. I converted to partial vegetarianism a while ago, and used Boca products to ease the transition. Then I stayed that way for five years. It’s probably time to try and take some more steps to improving my diet.
Margarine doesn’t count as “food” according to his definition.
I am interested in what you have to say about the “mostly plants” details. That part syncs up with pretty much everything I’ve otherwise heard.
I’m currently reading the full book; I’ll try to post some detailed thoughts after I finish and have some time.
Cool. My guess is that the snippets you’ve heard so far are the same kinds of buzzwords that were bothering me in the first half of the book. I think at least some of his motivation for that was to make the book more controversial and generate more buzz. Which, in terms of reaching the general population, may have been a good decision on his part. But I admit that if all you’re aware of is the controversial buzzwords, and there’s plenty of other good books you could be reading, ignoring a book with controversial buzzwords is also rational.
I have for a long time used a combination of directed search, breadth-first screening, and random samples.
Directed search is when I find one article on a subject, and I check its references to find other important articles on the subject, or use google or pubmed to find new references. Or when I click through Wikipedia links.
Breadth-first is when I pick some broad source and scan all the topics that come up on it. I read the complete index of every issue of Science magazine, to get an idea what is going on in a variety of fields. When I was in high school, I worked in the library, and looked at one shelf every day. By the time I graduated I had seen the title of every book in the library, and so I knew at least what the distribution of topics was.
I do random sampling with eMule: I will sometimes search for anything being shared that has some common word, like “the”, “for”, or “when”, in it. (Hint: Avoid “behind”.)
All of these work well for science. None of them work very well with literature or music.
runs off to test...
Not websites, but friends and colleges. I talk about the same topics, giving/receiving arguments, asking for sources (of course those are rare...). Sometimes one gem shows up. More often it is just a lossy, bidirectional knowledge-transfusion.
If such talks are not an option, and the interesting/important enough for me, I start googling. Very seldom, actually, because this means requiring loads of time. However, for serious stuff, nothing else helps.
I have basically two friends that I can talk to about things like this (I have other friends who are smart but who aren’t terribly interested in extensive, serious discussions). One of them is a fundamentalist Christian, so there’s a lot of disagreement, but the points of disagreement essentially boil down to completely different axioms that don’t leave much room for knowledge diffusion. In everything that ISN’T directly impacted by his religious beliefs, he pretty much either agrees with me, or disagreements between us stem from one/both of us being unaware of the facts, and as soon as we’ve done some research, we arrive at the same conclusions.
My other friend mostly ends up agreeing with me (or I with her) about things when it what actions should actually be taken. But oddly enough her thought process is completely different. She is completely frustrated with abstract hypothetical thought experiments, she doesn’t feel any need to have a moral system based on central axioms or even any dedicated examinations of what is right and why. So far as I can tell her morals are loosely based on “People have certain rights that are just inherent to the world,” and those rights are mostly unrelated and arbitrary. I do think that when you’re starting from a position of “maximize everyone’s preferences” you end up quickly arriving at “it’s beneficial to act as if people have certain inalienable rights”, and those rights are, for the most part, the same ones she believes in.
But notable exceptions occur, and it’s frustrating to me that so many elements of her morality are axiomatic and non-negotiable. For example, “right to privacy” is a big deal to her. I mostly agree with that. But a major point of disagreement is whether it’s reasonable for the government to have a database of fingerprints that are taken at birth. I don’t consider fingerprints “private.” You leave them everywhere. I have no close, intimate relationship with my fingerprints.
So for me, the question is “what is the probability that the government turns into a totalitarian state such that I’d actually be afraid of them having access to the general population’s fingerprints.” I know that we disagree strongly on what that probability, but even if we didn’t she opposes it on principle.
Not sure there is any lesson in those two anecdotes. Just thought it was interesting and somewhat relevant.