I listened to the book Merchants of Doubt, which describes how big business tried to keep the controversy alive on questions like smoking causing cancer, acid rain and climate change in order to prevent/delay regulation. It reports on interesting dynamics about science communication and policy, but it is also incredibly partisan (on the progressive pro-regulation side).[1]
Some interesting dynamics:
It is very cheap to influence policy discussions if you are pushing in a direction that politicians already feel aligned with? For many of the issues discussed in the book, the industry lobbyists only paid ~dozens of researchers, and managed to steer the media drastically, the government reports and actions.
Blatant manipulation exists
discarding the reports of scientists that you commissioned
cutting figures to make your side look better
changing the summary of a report without approval of the authors
using extremely weak sources
… and the individuals doing it are probably just very motivated reasoners. Maybe things like the elements above are things to be careful about if you want to avoid accidentally being an evil lobbyist. I would be keen to have a more thorough list of red-flag practices.
It is extremely hard to have well-informed cost-benefit discussions in public:
The book describes anti-regulation lobbyists as saying “there is no proof therefore we should do nothing”.
But the discourse of the book is not much better, the vibe is “there is scientific evidence of non-zero, therefore we must regulate”
(but hopefully the book is just underreporting the cost-benefit discussion, and it actually happened?)
Scientists are often reluctant to make noise about the issues they are working on, in part because they feel personally threatened by the backlash.
Even if fears about a risk become very widespread within the scientific community, it does not follow that governments will take actions in a timely manner about them.
Some things that the book does that greatly reduced my trust in the book:
It never dives into cost-benefit analyses. And when it goes near it, it says very weird things. For example, the book implies the following is a good argument: “acid rains harming bacteria in lakes should be taken very seriously because even if each bacteria is only worth $1, then that would be $10^24 of harm”.[2][3] It never tries to do post-mortem cost-benefit analysis of delaying regulation.
It is very left-wing partisan. The book makes 6 case studies, all of them making pro-regulation progressives feel good about themselves: it never discusses cases of environmental overregulation or cases of left-wing lobbyists slowing down right-wing regulation (both of which surely must exist, even if they are less common). It gives opposite advice to progressives and conservatives, e.g. it describes anti-regulation scientists as “being motivated by attention” while lamenting how the real climate scientists dislike the spotlights. It randomly dunks on free market capitalism with weak arguments like “in practice the axioms of free market economics are not met”. It is skeptical of technological solutions by default, even for relatively consensual things like nicotine patches.
The book doesn’t provide much evidence against the reasonable anti-regulation objections:
It doesn’t acknowledge the problem of most academics being left-leaning in the considered fields.[4] I don’t think it’s a severe issue for medicine and climatology, but not because of the book: the book provides very little evidence about this (it just says things like “even if scientists are socialists, the facts remain the same” without clearly spelling out why you would be able to trust socialist scientists but not scientists funded by big tabacco).
It doesn’t discuss how “let the free market create wealth and fear the centralization of power that state control creates” has historically been a decent heuristic.
Overall, I think the book contains some interesting stories, but I suspect it often paints an incomplete picture. It also fails to provide a recipe for how to avoid being an evil lobbyist,[5] or how to distinguish dishonest lobbyist arguments from reasonable anti-regulation arguments besides “trust established academia and don’t give a voice to scientists with slightly fewer credentials” (which seems like a decent baseline if you spend 0 effort doing investigations, but is probably not applicable if you plan to investigate things yourself or if you trust some people more than an established academia filled with left-leaning scientists).
I think the authors make other “math mistakes” in the book, such as saying that second order effect are effects which only matter for the second significant figure, and therefore always don’t matter as much as first order effects?
Not the exact quote, I did not manage to track it down in my audiobook or online. Would appreciate if someone had a text version of the book and was able to track the exact citation down.
I did not check this is actually true. I don’t know how strong this effect is for climatology and medicine. I suspect it’s strong for climatology. Would love some numbers here.
Which I think is something AI x-risk advocates should be worried about: a large fraction of AI x-risk research and advocacy is loud and mostly industry-funded (especially if you count OpenPhil as industry-funded), with views which are relatively rare in academia.
Gene Likens recalls one particularly frustrating moment, when he blurted out, “Fred [Singer], you’re saying that lakes aren’t valuable. They are economically valuable. Let me give you an example. Let’s say every bacterium is worth $1. There are 104 – 106 bacteria [ten thousand to a million] in every milliliter of water. You do the math.” Singer replied, “Well, I just don’t believe a bacterium is worth a dollar,” and Likens retorted, “Well, prove that it isn’t.” Twenty-six years later, Likens recalled, “It was the only time I ever shut him up.”
Left-vs-right is not the only bias that matters. Before the pandemic, I would have thought that virologists care about how viruses are transmitted. It seems, that they don’t consider that to be their field.
Given that virologists are higher status in academia than people in environmental health who actually care about how viruses are transmitted outside the lab, the COVID19 seems to have been bad. Pseudoscience around 6-feet distancing was propagated by government regulations. Even Fauci admits that there was no sound reasoning that supported the 6-feet rule.
Fauci also decided against using use money from the National Institute of Allergy and Infectious Diseases to fund studies about community masking as a public health intervention. You don’t need virologists to run studies about masking, so probably that’s why he didn’t want to give money to it.
While Fauci was likely more to the left, that did not create the most harmful biases in the policy response that didn’t want to use science to it’s fullest potential to reduce transmission of COVID19 but rather wanted to give billions to the Global Virome Project.
In another case, grid-independent rooftop solar installations are a lot more expensive than they would need to be. Building codes are made by a firefighter interest group in the US, and for firefighters it’s practical if the rooftop solar cells shut of when disconnected from the grid and as a result the pushed based on flimsy evidence for regulation that means that most rooftop solar in the US doesn’t work if the grid is cut off.
The question of whether you want grid-independent rooftop solar, is not one of left-vs-right but the biases are different.
Especially, today where many experts are very narrow in their expertise and have quite specific interests because of their expertise, thinking in terms of left-wing and right-wing is not enough.
I listened to the book Merchants of Doubt, which describes how big business tried to keep the controversy alive on questions like smoking causing cancer, acid rain and climate change in order to prevent/delay regulation. It reports on interesting dynamics about science communication and policy, but it is also incredibly partisan (on the progressive pro-regulation side).[1]
Some interesting dynamics:
It is very cheap to influence policy discussions if you are pushing in a direction that politicians already feel aligned with? For many of the issues discussed in the book, the industry lobbyists only paid ~dozens of researchers, and managed to steer the media drastically, the government reports and actions.
Blatant manipulation exists
discarding the reports of scientists that you commissioned
cutting figures to make your side look better
changing the summary of a report without approval of the authors
using extremely weak sources
… and the individuals doing it are probably just very motivated reasoners. Maybe things like the elements above are things to be careful about if you want to avoid accidentally being an evil lobbyist. I would be keen to have a more thorough list of red-flag practices.
It is extremely hard to have well-informed cost-benefit discussions in public:
The book describes anti-regulation lobbyists as saying “there is no proof therefore we should do nothing”.
But the discourse of the book is not much better, the vibe is “there is scientific evidence of non-zero, therefore we must regulate”
(but hopefully the book is just underreporting the cost-benefit discussion, and it actually happened?)
Scientists are often reluctant to make noise about the issues they are working on, in part because they feel personally threatened by the backlash.
Even if fears about a risk become very widespread within the scientific community, it does not follow that governments will take actions in a timely manner about them.
Some things that the book does that greatly reduced my trust in the book:
It never dives into cost-benefit analyses. And when it goes near it, it says very weird things. For example, the book implies the following is a good argument: “acid rains harming bacteria in lakes should be taken very seriously because even if each bacteria is only worth $1, then that would be $10^24 of harm”.[2][3] It never tries to do post-mortem cost-benefit analysis of delaying regulation.
It is very left-wing partisan. The book makes 6 case studies, all of them making pro-regulation progressives feel good about themselves: it never discusses cases of environmental overregulation or cases of left-wing lobbyists slowing down right-wing regulation (both of which surely must exist, even if they are less common). It gives opposite advice to progressives and conservatives, e.g. it describes anti-regulation scientists as “being motivated by attention” while lamenting how the real climate scientists dislike the spotlights. It randomly dunks on free market capitalism with weak arguments like “in practice the axioms of free market economics are not met”. It is skeptical of technological solutions by default, even for relatively consensual things like nicotine patches.
The book doesn’t provide much evidence against the reasonable anti-regulation objections:
It doesn’t acknowledge the problem of most academics being left-leaning in the considered fields.[4] I don’t think it’s a severe issue for medicine and climatology, but not because of the book: the book provides very little evidence about this (it just says things like “even if scientists are socialists, the facts remain the same” without clearly spelling out why you would be able to trust socialist scientists but not scientists funded by big tabacco).
It doesn’t discuss how “let the free market create wealth and fear the centralization of power that state control creates” has historically been a decent heuristic.
Overall, I think the book contains some interesting stories, but I suspect it often paints an incomplete picture. It also fails to provide a recipe for how to avoid being an evil lobbyist,[5] or how to distinguish dishonest lobbyist arguments from reasonable anti-regulation arguments besides “trust established academia and don’t give a voice to scientists with slightly fewer credentials” (which seems like a decent baseline if you spend 0 effort doing investigations, but is probably not applicable if you plan to investigate things yourself or if you trust some people more than an established academia filled with left-leaning scientists).
The book is strongly recommended by Stephen Casper, whose takes I usually like. So maybe my criticism is a bit too harsh?
I think the authors make other “math mistakes” in the book, such as saying that second order effect are effects which only matter for the second significant figure, and therefore always don’t matter as much as first order effects?
Not the exact quote, I did not manage to track it down in my audiobook or online. Would appreciate if someone had a text version of the book and was able to track the exact citation down.
I did not check this is actually true. I don’t know how strong this effect is for climatology and medicine. I suspect it’s strong for climatology. Would love some numbers here.
Which I think is something AI x-risk advocates should be worried about: a large fraction of AI x-risk research and advocacy is loud and mostly industry-funded (especially if you count OpenPhil as industry-funded), with views which are relatively rare in academia.
Here you go:
Thanks for tracking it down!
So I misremembered the exact quote, but I am not very far, at least in spirit? (There would be 10^(24 − 6) milliliters in 1000km² of 100m-deep lakes.)
Left-vs-right is not the only bias that matters. Before the pandemic, I would have thought that virologists care about how viruses are transmitted. It seems, that they don’t consider that to be their field.
Given that virologists are higher status in academia than people in environmental health who actually care about how viruses are transmitted outside the lab, the COVID19 seems to have been bad. Pseudoscience around 6-feet distancing was propagated by government regulations. Even Fauci admits that there was no sound reasoning that supported the 6-feet rule.
Fauci also decided against using use money from the National Institute of Allergy and Infectious Diseases to fund studies about community masking as a public health intervention. You don’t need virologists to run studies about masking, so probably that’s why he didn’t want to give money to it.
While Fauci was likely more to the left, that did not create the most harmful biases in the policy response that didn’t want to use science to it’s fullest potential to reduce transmission of COVID19 but rather wanted to give billions to the Global Virome Project.
In another case, grid-independent rooftop solar installations are a lot more expensive than they would need to be. Building codes are made by a firefighter interest group in the US, and for firefighters it’s practical if the rooftop solar cells shut of when disconnected from the grid and as a result the pushed based on flimsy evidence for regulation that means that most rooftop solar in the US doesn’t work if the grid is cut off.
The question of whether you want grid-independent rooftop solar, is not one of left-vs-right but the biases are different.
Especially, today where many experts are very narrow in their expertise and have quite specific interests because of their expertise, thinking in terms of left-wing and right-wing is not enough.