The Global Catastrophic Risks Institute conducted an anonymous survey of relevant experts on whether they thought COVID was more likely caused by a lab accident (aka lab leak) or zoonotic spillover. Their summary, bolding is mine:
The study’s experts overall stated that the COVID-19 pandemic most likely originated via a natural zoonotic event, defined as an event in which a non-human animal infected a human, and in which the infection did not occur in the course of any form of virological or biomedical research. The experts generally gave a lower probability for origin via a research-related accident, but most experts indicated some chance of origin via accident and about one fifth of the experts stated that an accident was the more likely origin. These beliefs were similar across experts from different geographic and academic backgrounds.
The experts mostly expressed the view that more research on COVID-19’s origin could be of value. About half of the experts stated that major gaps still remain in the understanding COVID-19’s origin, and most of the other experts also stated that some research is still needed. About 40% of experts stated that clarity on COVID-19 origins would provide a better understanding of the potential origins of future pandemics. Given clarity on COVID-19’s origin, experts also proposed a variety of governance changes for addressing future pandemics, including measures to prevent initial human infection, measures to prevent initial infection from becoming pandemic, and measures to mitigate the harm once the pandemic occurs.
The vast majority of the experts express the belief that a natural zoonotic event will likely be the origin of the next pandemic.
The experts also provided a set of clear recommendations for preventing, preparing for and responding to future pandemics, which generally align with many previous studies.
Link to the main report is here, and link to their (much longer) methodological and analytical annex is here.
[EDIT: I currently think there are enough problems with the survey that they should be mentioned alongside the results.
Firstly, the sample seems to have been based on personalized outreach, rather than mass emails. This runs the risk of selection bias. [EDIT 2: I’m told that ‘personalized outreach’ looks more like “sending individual emails to everyone on a big list” than “the authors emailing their friends”.] Also, some participants were recruited by recommendations from other participants, which may reduce the effective sample size by over-drawing from pools of people who agree with each other. [EDIT 2: But this was mitigated to some degree by disciplinary and geographic diversity.] [EDIT 3: See this comment and replies by one of the authors of the report on the survey.]
Secondly, the survey asked people if they were familiar with a few different papers that analyse the evidence for or against a lab leak. They also included a fake paper to see how often people lied about their familiarity. A third of the sample claimed to be familiar with a fake study, more than were familiar with one relevant piece of evidence, the DEFUSE proposal, but much fewer than were familiar with most other pieces of relevant evidence.]
There is a risk that such a survey includes Peter Daszak and his friends and collaborators, who may have been the ones responsible for the covid-19 lab leak.
Or, that it includes people who have been influenced by Daszak via spreading false information and so on, or are collaborators or are otherwise incentivized to lie for personal and career advancement. The field is small enough that personal relationships are essential to getting funded, hired, published and so on.
Plus, there are effects like information cascades and respectability cascades; everyone updates towards the more respectable opinion rather than the true one. And surveys like this are part of that effect.
I personally think that the chance that covid-19 was created in a lab in Wuhan is exceptionally high, perhaps 93%, and there are various skeptical experts who think it is now beyond reasonable doubt that the Wuhan Lab created covid-19.
As such I would label this entire exercise as mistake, akin to asking a sample of organized criminals whether organized crime is a problem in their area, and having 90% of them saying it isn’t.
Note that Ralph Baric, a prominent scientist who has done gain of function research on SARS-like betacoronaviruses and was on the DEFUSE proposal lead by Daszak, signed an open letter with Jesse Bloom, Alina Chan, and others calling for further investigation of the lab leak hypothesis than had been done previously. So Daszak is apparently not so powerful that he’s able to silence the whole field.
People often parse information through an epistemic consensus filter.
They do not ask “is this true”, they ask “will others be OK with me thinking this is true”. This makes them very malleable to brute force manufactured consensus; if every screen they look at says the same thing they will adopt that position because their brain interprets it as everyone in the tribe believing it.
(Loosely quoted from a famous 4chan greentext)
Daszak—who likely killed 20 million people a few years ago—is using “brute force manufactured consensus” to hide his crimes, and the Global Catastrophic Risks Institute is unwittingly helping him. This report doesn’t give any reasons why covid-19 wasn’t a lab leak. It’s just people posting an unjustified and false opinion, which other people then see and most (but not all) succumb to the “brute force manufactured consensus”. They likely got that false and unjustified opinion from a previous version of this same attack; high-status manufactured consensus spreads like a virus.
I do not think this kind of exercise helps to get at the truth or to reduce global catastrophic risks; in fact it increases them because it makes this “brute force manufactured consensus attack” easier to run.
Don’t ask people with status to post opinions, ask them to give gears-level explanations and justifications. Make them commit to cruxes. Make them give probabilities for specific testable sub-questions.
How is this a response to my point, that you can apparently be a virologist who has worked with Daszak and still publicly disagree with him?
It’s a response because I am explaining how I think this works; most (but not all!) people are updating on the consensus opinion about what’s high status, and then of the remainder some are rationally lying, and there are some small number of people in the field who are both rational and honest.
I think you grossly underestimate how hungry scientists are to prove each other wrong. This is part of how you build status to begin with. Yes, there are collaborative relationships, but there are also a great many adversarial relationships. There is no top-down hierarchy, so silencing dissent in this manner is unavailable.
I do think some degree of self-censorship occurs, absolutely. Are there biases, sure. But I find the claim that any given person is so influential in epidemiology that there is a conspiracy of silence lasting quite this long rather absurd.
FWIW much of the sample is epidemiologists who don’t think it was a lab leak. The survey was also anonymous, so people could dissent from the pro-zoonosis consensus without fear of retribution (as many did).
But I don’t think this is how normies work. They don’t first believe the truth, then strategically decide to lie about it to keep in good standing with the high-status people, then when you ask them anonymously revert to their true belief.
No, they don’t look for truth at all. They look for what the high-status tribal belief is, then they go and believe that honestly and earnestly.
This is why we needed a 500k+ word sequence of blog posts to teach people what truth even is.
In addition to Roko’s point that this sort of opinion-falsification is often habitual rather than a strategic choice that a person could opt not to make, it also makes strategic sense to lie in such surveys.
First, the promised “anonymity” may not actually be real, or real in the relevant sense. The methodology mentions “a secure online survey system which allowed for recording the identities of participants, but did not append their survey responses to their names or any other personally identifiable information”, but if your reputation is on the line, would you really trust that? Maybe there’s some fine print that’d allow the survey-takers to look at the data. Maybe there’d be a data leak. Maybe there’s some other unknown-unknown you’re overlooking. Point is, if you give the wrong response, that information can get out somehow; and if you don’t, it can’t. So why risk it?
Second, they may care about what the final anonymized conclusion says. Either because the lab leak hypothesis becoming mainstream would hurt them personally (either directly, or by e. g. hurting the people they rely on for funding), or because the final conclusion ending up in favour of the lab leak would still reflect poorly on them collectively. Like, if it’d end up saying that 90% of epidemiologists believe the lab leak, and you’re an epidemiologist… Well, anyone you talk to professionally will then assign 90% probability that that’s what you believe. You’d be subtly probed regarding having this wrong opinion, your past and future opinions would be scrutinized for being consistent with those of someone believing the lab leak, and if the status ecosystem notices something amiss...?
But, again, none of these calculations would be strategic. They’d be habitual; these factors are just the reasons why these habits are formed.
Answering truthfully in contexts-like-this is how you lose the status games. Thus, people who navigate such games don’t.
I’ve seen a preprint which is very close to being a smoking gun, although its claim needs to be independently reproduced. The preprint says that when they have filtered the standard sequence database by the date of submission (keeping only sequences which were submitted before the start of the pandemic (as opposed to being marked as discovered before the start of the pandemic)), then there is a clear match with a particular 2008 PNAS paper and the sequence published in connection with that 2008 paper.
If this reproduces, this would make it very likely that it has been a lab leak, and that moreover, the research involved has been reproducing the synthetic virus published in 2008 (and would not be possible without that 2008 publication). For obvious reasons, the paper has not been welcomed at all by any established outlets, since it makes both US and China scientific establishments and their collective practices look really bad. Nevertheless, here it is: https://www.researchgate.net/publication/353031350_The_possible_laboratory_origins_of_SARS-CoV-2_the_likelihood_of_a_subsequent_deadlier_COVID_pandemic_and_necessity_to_introduce_blockchain_practices_for_verifying_and_tracking_scientific_data
To make sure those claims are correct someone would need to independently reproduce its findings. It’s not all that straightforward, because people running the sequence database in question has turned the ability to filter by the date of submission off since then (which might has been done for related or unrelated reasons, I would not know for sure). However, I think one can still download the whole thing and run the needed searches locally and let people know whether claims do reproduce or not, although it would require some effort.
The 2008 paper which is supposedly involved is this one: https://www.pnas.org/doi/10.1073/pnas.0808116105
I don’t think “clear match” is a good term. Whether something is a clear match or not does not depend on other sequences that were published. If you sort them, you can see which of the published sequences is the “nearest match”, but that’s a different concept than “clear match”.
With that nearest match having only 74% identity, that’s a lot of distance to the actual SARS-Cov-2 virus.
To me, it looks like a low-quality paper that doesn’t deserve much attention and is far from anything worth being called a smoking gun.
You don’t really need that. You can just look at the sequences under discussion.
My impression is different. If what they say does reproduce, then to me this would look likely that the pandemic has been the result of research which has been partially reproducing this particular published 2008 work.
In any case, this would need to be publicly discussed, not hidden under covers (I know some details of how decisions whether to accept this for publication have been made in some of the cases and it has never been about the quality of the paper, when it has been explicitly discussed it has always been about possible repercussions for a journal in question and for its editors (and when it has not been discussed, it has always been a technicality in the style of “in this particular case it looks like the venue profile is not a fit for this text”, even if the venue has published on similar topics)).
To summarize the claim:
this is the best match among all sequences ever submitted before the pandemic,
and the critical region match is very high,
and the critical region match is not high for any other sequences for which the overall match is approaching this one.
So, the claim is not just about this being the best match among all sequences ever submitted before the pandemic, it’s quite a bit more than that.
It is rather obvious that Covid is not derived directly from this virus (the difference is too big), but it does look to me and to many other readers of this preprint, although not to everyone, that it’s likely that people who have synthesized the original Covid have been looking at the published 2008 sequence while doing their work.
All that is conditional on the claim actually reproducing (which is why it is very annoying that the database admins has made it more difficult to try to reproduce it, either deliberately or accidentally).
There is enough here to discuss publicly, I think (and yes, enough room to disagree about the interpretation of the findings).
Why are you using vague terms like “very high” instead of being specific about the numbers you consider to be “very high”? It would be much easier to follow your claim if you would be more specifc.
Why? How do you expect that synthezing process to look like? What motivation do you imagine for it as a research project?
Why would they synthezise the whole genome instead of just taking one viruses they had in their lab and insert the mutations they want to study? Inserting specific mutations is much easier than synthesizing the whole thing.
It’s a visual illustration, no? Visually this looks rather strong (Figures 1 and 2 on pages 16-17).
I don’t think they had the virus. They had what’s was published, not the materials.
And I do think that China might have enough research manpower to just routinely reproduce all published findings of this kind of experiments if they want to (I don’t know if they actually do that; I would not be too surprised if they do that as a routine though; this depends on what are their actual policies; I have no means to investigate that).
(So, I would presume they would have reproduced it earlier, and were using the results of that reproduction for various things as needed without thinking much, and one of those subsequent things leaked.)
The visual match can quite easily show you that one match is stronger than another but they don’t tell you how good a match in question actually happens to be. There are ways to measure whether something is a good match with numbers.
The idea that a country would just spend that much research capital and do that without it leaving any trace in their research publications and other public communication seems farfetched.
Yes, an independent reproduction would also evaluate if their methodology is actually good in this sense (I can imagine all kinds of methodological “underwater stones”).
I did not mean to give an impression that I had made up my mind about the outcome of this potential further exploration. I had made up my mind that it’s worth further exploration, but I would not predict the results. Unfortunately, it is not all that easy to arrange (we do know that neutral prior here is important, rather than someone heavily leaning towards one side doing it, because there is always room for pushing results towards this or that direction; for example, I’ve spent too much “quality time” with this paper to be considered a fully neutral person, although I would certainly make an effort to avoid the bias if I were to do this work; then one might be unsure how safe it would be to publish on this, even today, and so on).
They would report to the government (if the order to reproduce things comes from the government). The government would decide what to make public and what to keep for more restricted use. It’s very natural (especially if the subject is potentially “dual-use”, or, at least, is considered relevant to national defense).
This might have already been covered somewhere, but I’m curious what makes you think COVID-19 was created in a lab and not a natural virus leaked while they were studying it.
Update: Roko wrote a whole post about this.
Sorry, I misread this.
I don’t have a strong opinion. I think it’s plausible that it was a lab leak of a natural virus, though I will note that the more technical people are more skeptical of this, claiming that covid-19 was unusually good at spreading through humans even from the start, which is unlikely for a fully natural spillover.
If I knew more about the details of virology, I would have a stronger opinion.
Well, be Bayesian about it.
Create a virus in a natural spillover event; what is the chance that that spillover happens within a few miles of the lab that is studying the virus?
I think you’re answering a different question from the one Brendan asked.
You’re answering “Why do you think COVID-19 escaped from a lab?”.
Brendan was asking “Conditional on COVID-19 having escaped from a lab, why do you think it was created there rather than being a natural virus they were studying in that lab?”.
I don’t have a strong opinion on that question.
Aside from the fact that you’re answering a different question from Brendan’s, this argument seems like it involves some assumptions that are not known to be correct.
Isn’t the right version of your question “Create a virus in a natural spillover event; what is the chance that that spillover happens within a few miles of a lab that is studying similar viruses?”?
The answer to that might be “fairly large”, if e.g. it happens that there are virus labs and likely spots for natural zoonotic spillover located near to one another. Which is in fact the case in Wuhan, no?
(I don’t know how well the details of the case fit the “escape from the Wuhan virus lab” and “zoonosis at the Wuhan wet market” hypotheses. Maybe they’re a better fit for the former than for the latter. But that’s a very different sort of argument from “there’s a virus lab that was studying coronaviruses near to where COVID-19 was first seen in humans”.)
But WIV was the only lab in China studying this virus, whereas the Wuhan wet market is nothing special—they are all over the country.
So if spillovers happen at random in wet markets, the probability of getting the market closest to the lab in the whole of China which is HUGE, AND ALSO hitting the exact temporal window where they are doing this particular research is very small. There was nothing stopping a natural spillover happening in the 1980s, 1990s, 2000s, etc, and all of those would have missed this research as it was not technologically possible then.
You’re saying “this virus” again when what’s actually known is that WIV was studying coronaviruses, not specifically that it was studying SARS-COV-2.
(If it turns out that WIV was studying SARS-COV-2 specifically before it started infecting humans then yes, that would be very strong evidence in favour of lab leak theories.)
Anyway: yes, I do agree that the fact that SARS-COV-2 first got into humans somewhere rather near a lab that was studying similar viruses is substantial Bayesian evidence that it got into humans via that lab. But the exact same thing that makes it substantial evidence (there aren’t many such labs, whereas there are many opportunities for natural zoonosis which could happen in a wider variety of places) also means that the prior is low.
So the question is roughly “how dangerous do you think, on priors, a single WIV-like lab is, compared to a large wet market?”. (“Dangerous” meaning probability of releasing coronaviruses into the human population.) If, before hearing about SARS-COV-2, you would have thought WIV was about as likely to release coronaviruses into the human population as the Wuhan wet market, then after hearing about SARS-COV-2 you should think the probability it was a lab leak is about 50%. (And then potentially modify that substantially depending on all the details that we’re ignoring in this discussion, which might give more concrete evidence to distinguish the hypotheses.) Etc.
[EDITED to add:] I’ve now seen your post about this; I agree that the DEFUSE thing seems like highly relevant evidence (but haven’t looked into the DEFUSE proposal myself to check whether I agree with what you say about it). If it’s correct that WIV is known to have been working on something much more specifically matched to SARS-COV-2 then that dangerousness ratio looks quite different (because something so much more specific is correspondingly less likely to occur as a natural zoonosis).
Well, yes, DEFUSE is helpful here.
But I concluded that covid-19 was a lab leak in early 2020 purely based on the geographical coincidence of a novel coronavirus appearing exactly on the doorstep of China’s first ever BSL-4 certified lab at WIV.
There are two one-off events—China getting its first BSL-4 certified lab, and China getting the largest pandemic of the past hundred years. And they happened within the same population blob of 7 million people, which is about 0.5% of the total population of China.
There were already some other special things about WIV that we knew in 2020, like the affiliations of coronavirus researchers, and WIV was the top one. WIV was THE place to study coronaviruses in China. See, e.g.
https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7148667/
and especially
https://scholar.google.com/scholar?q=coronavirus+china&hl=en&as_sdt=0%2C5&as_ylo=2010&as_yhi=2019
Now DEFUSE and the Ecohealth stuff is additional evidence. DEFUSE specifically links WIV to
the Yunnan caves
gain of function using a Furin Cleavage Site
uniformly spaced recognition sites and BsmBI (BsmBI is in the DEFUSE proposal)
All this stuff collecting at WIV which uniquely fits the virus adds further nails to the coffin, though of course there is always the danger of confirmation bias. However, the researchers who identified the uniformly spaced recognition sites did so before DEFUSE was unearthed so they didn’t know that BsmBI was waiting for them in there.
It’s over. Daszak & co killed 27 million people.
The DEFUSE proposal that you linked to doesn’t (so far as I can tell) say anything about where the furin cleavage site work would be done. OP here includes an image that seems to show a Word document or something with a comment about that, but it isn’t obvious to me where it’s from or how it’s known to be genuine or anything.
The “uniformly spaced recognition sites and BsmBI” thing (at least, the instance of it that I’ve found) looks rather sketchy and unconvincing to me, though I’m not really competent to evaluate it (are you?). It’s possible that what I’m looking at isn’t what you’re referring to; I’m talking about a post on Alex Washburne’s Substack where he draws attention to mention in the DEFUSE proposal of “reverse genetic systems” and “infectious clone technology” (though so far as I can see neither of these is actually mentioned in the proposal itself), claims (I do not know with what evidence) that using these methods would produce unusually regularly spaced instances of certain genome subsequences that are targeted by BsmBI or a similar enzyme, and claims that the SARS-CoV-2 shows such unusually regularly spaced instances.
But (1) unless there’s something I’m missing this “specifically links WIV to” those methods only in a very weak sense (e.g., the DEFUSE proposal doesn’t in fact say that they are going to use those methods, or that it would be done at the WIV), (2) Washburne doesn’t provide any support for his claim that researchers using this technique would in fact make the relevant segments unusually uniform in length, and (3) nor does he seem to give any details of the analysis that supposeedly shows that SARS-CoV-2 has such unusually uniform segments. He makes some claims about earlier drafts of the DEFUSE proposal supposedly obtained by FOIA requests, which if correct go some way to filling these gaps a bit, but if he actually shows those or gives evidence that they’re real then I haven’t seen it.
(Note: I find the style of Washburne’s writing very offputting; it pattern-matches to e.g. creationists crowing about their nonsensical statistical arguments. Lots of expressions of triumph, pointlessly flowery writing, that sort of thing. Of course that isn’t strong evidence that Washburne is wrong in the same sort of way as the creationists are, but I find it does strongly incline me to skepticism. It’s odd that, when arguing that something previously dismissed as a conspiracy theory is probably true, it doesn’t occur to him to try not writing like a conspiracy theorist.)
Daszak’s comments say that much of the work could be done at WIV which could include the FCS. The comment is genuine AFAIK but you’ll have to chase it up yourself.
The work on BsmBI is somewhat compelling because the people who suspected it published before DEFUSE was unearthed, and then DEFUSE was found to contain an order for BsmBI. So, they sort of predicted this. And there are reasons to make the virus out of relatively uniformly long segments—it’s convenient, the tools and techniques have maximum lengths they can handle, and you want to minimize the total amount of work. So you use roughly equal length segments. However this is a fairly complicated series of claims and I don’t see it as being the big win for lab-leak—the big win is DEFUSE itself.
But zoom out a bit: we can just notice that all of this stuff only collects around Wuhan, WIV and associated facilities like State Key Laboratory of Virology at Wuhan University.
There’s no equivalent of this in all the other cities in China and I am pretty sure that if you look through the top 20 cities by population you won’t find something like DEFUSE. I tried this a bit with chatGPT, Google Scholar, Google Search etc. There simply are not 50 other Daszaks out there doing GoF bat coronavirus research in every other city in China. Wuhan is THE PLACE where this happens.
Why does that mean the prior is low?
I see no reason why I would assume that the prior for a lab to leak a virus with airborne transmission when they handle it under biosafety level II which is not designed to prevent airborne transmission is low.
The prior for any given newly-emerged virus being a natural zoonosis rather than a lab leak is higher when there are fewer labs to leak.
I agree that the prior for a leak happening from any given lab at any given time doesn’t depend on how many labs there are, of course.
How many virus strains is the lab studying? If the lab is studying 50-90% of flu virus strain it would not be strange for random flu virus that appeared in some area close to it to be studied there.
But this isn’t a random flu virus. It’s a once-in-a-century pandemic!
But we don’t care about random flu virus. We only track pandemic.
Furthermore random pandemic virus could happen in rural areas but more likely to turn into pandemic when they happen in crowded city. The more crowded the higher the pandemic chances.
How many lab similar to Wuhan in crowded cities vs how many crowded city without lab should be taken into account
None. Wuhan is the only BSL4 lab in China, and it is the only place that did bat coronavirus gain of function research. And Shi Zhengli’s group at WIV is the premier group in China that studies bat coronaviruses.
Wikipedia says there is another BSL-4 lab in Harbin, Heilongjiang province. (Source is an archived Chinese news site) Is that incorrect?
That is correct
In Feb 2020 Anthony Fauci convened a bunch of virologists to assess SARS-CoV-2 origins. The initial take from the group (revealed in private Slack messages via FOIA requests from 2023) was this was likely engineered. In Kristian Andersen of Scripps Research’s view, it was “so friggin likely because they were already doing this work.”
The same month, Fauci held an off-the-record call with the group. After that, everyone’s tunes changed and shortly after (in a matter of weeks) we got the Proximal Origins paper, with Kristian Andersen doing a 180 as the lead author. The paper posits that there is “strong evidence that SARS-CoV-2 is not the product of purposeful manipulation.” I encourage you to read the paper to determine its merits. Their evidence as I understand it is a) the structure of the spike protein is not what a computer would have generated as optimally viral, and b) pangolins. Pangolins were ruled out as carriers shortly after the paper’s release. (a) can be dismissed—or at least mitigated—by the fact that serial passage can naturally develop what a computer may not. Andersen’s Scripps Research coincidentally got a multi-million dollar grant shortly after publishing Proximal Origins, but again, that’s merely coincidence.
Some in the comments seem stuck on the fact this virus could have been obtained in the wild, and thus is zoonotic in origin. That is ignoring the substantial work the Wuhan lab undertook to take natural viruses and create chimeric viruses that were optimized for human contagiousness.
Fauci took the Proximal Origins paper on his circuit of 60 Minutes interviews, NYTimes podcasts, and Congressional testimonies, declaring, “the leading virologists say this was most likely of natural origin”.
This rhetoric undoubtedly has a massive chilling effect on any “experts” who would otherwise posit that this could be lab origin. The High Minister of Science has declared it was zoonotic, definitive proof will probably never be established either way, so you better be on the side of the High Minister of Science.
If there were an omniscient arbiter of truth that could make markets on this issue, I would take lab leak >50% in a heartbeat, and have it closer to 85%. Alas, there never will be such an arbiter, and we’ll have to rely on the experts who are heavily reliant on government research grants, gain of function research as the way of the future, and generally not rocking the boat.
Do you think this chilling effect extends to other countries where Fauci has no presence?
Yes, by virtue of the alliance with the “top virologists”.
I observe that literally every country in the world chose not to do challenge trials in 2020, which could have sped up the vaccine rollout by around 8 months and prevented a great deal of the deaths in those countries. For hundreds of countries to all do the same thing here looks to me exceedingly like conformity (starting with some trend-setter, which I expect is the US). So I think that beliefs like this can quite easily explained by conformity.
Hundreds seems like the wrong sample size, more like around a dozen? Realistically, I would have thought that most countries probably don’t have the affordance to distribute vaccines much earlier.
Also worth noting that Russia did something pretty aggressive with respect to vaccine roll out which I think looks pretty good from a public heath perspective in retrospect. (I think they didn’t do full clinical trials prior to roll out.)
Fauci is not the only person who created chilling effects. The Chinese also did their best to discourage people from believing in the lab leak hypothesis and there are many others who had their own reasons for discouraging the belief as well.
This reminds me of a passage in Richard Feynman’s memoir “What do you care what other people think?”. Four pages into the chapter Gumshoes, (page 163 in the Unwin Paperback edition):
Then this business of Thiokol changing its position came up. Mr. Rogers and Dr. Ride were asking two Thiokol managers, Mr. Mason and Mr. Lund, how many people were against the launch, even at the last moment.
“We didn’t poll everyone,” says Mr. Mason.
“Was there a substantial number against the launch, or just one or two?”
“There were, I would say, probably five or six in engineering who at that point would have said it is not as conservative to go with that temperature, and we don’t know. The issue was we didn’t know for sure that it would work.”
“So it was evenly divided?”
“That’s a very estimated number.”
It struck me that the Thiokol managers were waffling. But I only knew how to ask simpleminded questions. So I said, “Could you tell me, sirs, the names of your four best seals experts, in order of ability?”
“Roger Boisjoly and Arnie Thompson are one and two. Then there’s Jack Kapp, and, uh … Jerry Burns.”
I turned to Mr. Boisjoly, who was right there, at the meeting. “Mr. Boisjoly, were you in agreement that it was okay to fly?”
He says, “No, I was not.”
I ask Mr. Thompson, who was also there.
“No. I was not.”
I say “Mr. Kapp?”
Mr. Lund says, “He is not here, I talked to him after the meeting, and he said, ‘I would have made that decision, given the information we had.’”
“And the fourth man?”
“Jerry Burns. I don’t know what his position was.”
“So,” I said, “of the four, we have one ‘don’t know,’ one ‘very likely yes,’ and the two who were mentioned right away as being the best seal experts, both said no.” So this “evenly split” stuff was a lot of crap. The guys who knew the most about the seals—what were they saying?
That is the end of that section of the chapter and Feynman turns to the infra-red thermometer and the temperatures on the launch pad.
That was my introduction to this aspect of bureaucratic infighting. The bureaucrat asks his technical experts, the one closest to the issue. If he gets the answer that he wants, it is accepted. If not, he widens the pool of experts. Those too close to the issue are at risk of ignoring the social cues to the desired answer, but the wider pool of experts can be more flexible at responding to the broader social context. Then the bureaucrat gets to take an unweighted average (that is not weighting the original experts more highly). Which boosts the probability of getting the desired answer and reduces the probability of getting the correct answer.
Back in 1988 this was perhaps a busted technique. But that was many years ago. The notion of broadening your survey of experts seems to be back in fashion.
A better headline would be: “85% of experts are not willing to answer a survey about whether or not they think COVID-19 was a lab leak”
Imo mildly misleading. I expect large parts of the 85% to just not have read their mails, or to have been too busy to answer what may look to them like a mildly useful survey.
I agree that it isn’t perfect, but it’s important information that should not be left out.
Indeed, but my impression is that low response rates are the default outcome when you mail around a survey like this.
(It’s natural to wonder about bias. My feeling is that people who think COVID-19 was a lab leak are probably more, not less, likely to want to answer an anonymous survey that gives them a chance to say so.)
I think you have to distinguish between people who believe that it was a lab leak and people who believe that it’s good that the public believes that it’s a lab leak.
If I would be a virologist who told everyone in 2020 that all people who believe in the lab leak are conspiracy theorists, but updated towards the lab leak being the probable outcome, I would feel shame about that and minimize my exposure to the topic and not feel happy to fill out surveys about it.
A relevant tweet from Nate Silver on the methodology used to conduct the survey:
The tweet has some screenshots of relevant parts of the paper
This is a misunderstanding as I discuss I my comment below. The respondents were selected from a list according to the criteria set out. This doesn’t mean the authors emailed their friends
If you just want the bottom-line number (emphasis mine):
This isn’t much of an update; talking about lab leak was effectively banned from polite society during 2021, and most credentialed AI experts today still don’t think AI is much of a risk. This study is more of a measure of what to expect from credentialed people, who were generally slow to update on X-risk related topics until too late (nuclear as well as AI), and at critical thinking in general.
It’s been a couple of years since 2021, so you might think that if there were persuasive reasons to believe in the lab leak theory, experts would know about them.
We are talking about experts in virology or other bio security-relevant domains. Those aren’t necessarily experts on all the evidence about the origin of COVID19.
The personalised outreach mentioned just means that the respondents were initially sent a stock email and then when they didn’t respond, they were sent a more personalised message. It doesn’t meant that the surveyors emailed their friends. The survey was based on mass outreach from a list from professional societies
Snowballing contacts does introduce a risk of bias but that is mitigated by the disciplinary and geographic spread in the target sample. Respondents in non developed countries gave a higher chance of zoonosis, so the prospect that the survey was biased because it was sent to eg Kristian andersen who then recommended people who knew favoured his opinion seems low.
It is true that the survey showed low familiarity with the relevant literature. First, this is an interesting finding in itself. Second, in many expert polls the field experts may not have read much of the literature on some specific question.eg this is likely true of the igm poll of economists, which is nevertheless useful.
Competing claims have been made about what virologists in general actually think about this topic. We now have some information on this
(Casual readers may not realize that John Halstead was one of the co-authors of the report on this survey)
Is there any chance you guys could share information about the trees of who recommended who, to help get a sense of how big this bias could be? Like, how large was the largest recommendation chain, what fraction of people were recommended vs initially contacted, etc?
There were six people of 168 respondents who came via people some of the authors already knew or snowballing. I don’t think this would have much effect on the overall picture of the results
Correction my apologies. Apparently 30 respondents were snowballed from 17 others. We’ll look into how this affects the results
Thanks!
Okay, so this is another instance of plenty of high-status people with a lot to lose declining to say they believe something that would get them fired and/or ostracized. Do you believe it has any bearing on whether or not it is true?
It was an anonymous poll so this would not have an effect
People used to self-censoring still often self-censor when they don’t have to. Anonymity should have changed the balance, though.
I think it has some bearing, partly because I’m pretty sure you’re wrong about whether they would be fired or ostracized. For example, I think your theory is confused by how many people said that lab leak was more likely than zoonosis.
A bunch of people in the comments section are skeptical that we should care about the consensus of experts on this question. One thing I’m curious to get people’s opinion on: late last year, Rootclaim did a series of debates with Peter Miller on whether COVID was a gain-of-function lab leak or a zoonotic spillover, you can watch the videos here. Two judges were mutually agreed upon, and for each judge that’s convinced one way or the other, the loser (according to that judge) has to pay the winner $50,000. As a result the debates were pretty extensive—they went for a total of 17 hours, and the judges were pretty engaged, including asking written questions between rounds. The judges haven’t released their decisions yet, but they will later this month.
For people who are inclined to disregard this survey: if the judges rule in favour of a zoonotic origin, would that count as relevant evidence in favour of zoonosis? Alternatively, if they rule in favour of a GoF lab origin, would that count as relevant evidence in favour of a lab leak?
Zoonotic will win this debate because Peter outclassed Saar on all fronts, from research/preparation to intelligibly engaging with counterclaims and judge’s questions.
Saar seemed too focused on talking his book and presenting slides with conditional probability calculations. He was not well-versed enough in the debate topics to defend anything when Peter undercut a slide’s assumptions, nor was he able to poke sufficient holes in Peter’s main arguments. Peter relied heavily on case mapping data, and Saar failed to demonstrate the ascertainment bias inherent to that data. He even admitted he did no follow-up research after the initial presentation.
I get the sense Saar either thought lab leak was so self-evident that showing the judges his probability spreadsheet would be a wrap, or he was happy to pony up $100k just to advertise Rootclaim. Maybe both.
For those reasons the Rootclaim verdict doesn’t seem like a proper referendum on the truth of the matter. But I would be more sympathetic to people updating toward zoonotic on the basis of having watched that debate, rather than on the basis of these survey results.
Metaculus currently puts the odds of the side arguing for a natural origin winning the debate at 94%.
Having watched the full debate myself, I think that prediction is accurate- the debate updated my view a lot toward the natural origin hypothesis. While it’s true that a natural coronavirus originating in a city with one of the most important coronavirus research labs would be a large coincidence, Peter- the guy arguing in favor of a natural origin- provided some very convincing evidence that the first likely cases of COVID occurred not just in the market, but in the particular part of the market selling wild animals. He also very convincingly debunked a lot of the arguments put forward by Rootclaim, convincingly demonstrated that the furin cleavage site could have occurred naturally, and poked some large holes in the lab leak theory’s timeline.
When you have some given amount of information about an event, you’re likely to find a corresponding number of unlikely coincidences- and the more data you have, and the you sift through it, the more coincidences you’ll find. The epistemic trap that leads to conspiracy theories is when a subculture data-mines some large amount of data to collect a ton of coincidences suggesting a low-prior explanation, and then rather than discounting the evidence in proportion to the bias of the search process that produced it, they just multiply the unlikelihood- often leading a set of evidence so seemingly unlikely to be a cumulative coincidence that all of the obvious evidence pointing to a high-prior explanation looks like it can only be intentionally fabricated.
One way you can spot an idea that’s fallen into this trap is when each piece of evidence sounds super compelling when described briefly, but fits the story less and less the more detail about it you learn. Based on this debate, I’m inclined to believe that the lab leak idea fits this pattern. Also, Rootclaim’s methodology unfortunately looks to me like a formalization of this trap. They really aren’t doing anything to address bias in which pieces evidence are included in the analysis, and their Bayesian updates are often just probability of a very specific thing occurring randomly, rather than a measure of their surprise at that class of thing happening.
If the natural origin hypothesis is true, I expect the experts to gradually converge on it. They may be biased, but probably aren’t becoming increasingly biased over time- so while some base level of support for a natural origin can be easily explained by perverse incentives, a gradual shift toward consensus is a lot harder to explain. They’re also working with better heuristics about this kind of thing than we are, and are probably exposed to less biased information.
So, I think the Rationalist subculture’s embrace of the lab leak hypothesis is probably a mistake- and more importantly, I think it’s probably an epistemic failure, especially if we don’t update soon on the shift in expert opinion and the results of things like this debate.
The main shift in expert opinion I see is that in 2020 those experts said that everyone speaking about the lab leak hypothesis is a conspiracy theorist to now being more open about the possibility of a lab leak. We also saw some experts like those at the Department of Energy and FBI to switch to believing the lab leak is the most likely explanation.
I’m not going to watch 17 hours of debate over this, but tentatively, I’d say yes, one side convincing mutually-agreed-upon expert judges would be evidence, because I think the judges have already paid most of the reputational cost for participating at all.
Rootclaim’s page is displaying a 89% chance that covid-19 was a lab escape plus 4.5% that it was a bioweapon
https://www.rootclaim.com/analysis/What-is-the-source-of-COVID-19-SARS-CoV-2
I think this is evidence, but weak evidence—it updates me more on “Rootclaim isn’t great at debates” than it does on the underlying issue. (Like, how much should William Lane Craig winning his debates update me on theism?)
I think if I started off at 90% confidence of lab leak, Rootclaim losing wouldn’t bring me below 80% confidence of lab leak. Plausibly Peter Miller’s arguments contain defeaters for my specific beliefs, and going thru the debate would bring me much lower, but I don’t yet have that sense from the summaries I’ve seen.
Partly I think that this debate format was way higher-quality than most formats I’ve seen, including in the domain of theism vs atheism. I also think that the answer is going to depend on whether or not your reasons for atheism are basically the same as Craig’s opponents’ - if they are, then I think it actually should somewhat (at least in a format where the winner is picked correctly), but if they aren’t, then it probably shouldn’t much at all.
This post makes me thankful the scientific method doesn’t have a step for “survey a bunch of experts” or “do an anonymous opinion poll.”
I find very little to trust or value in a broad opinion polls about contentious, politically charged issues if the goal is to enlighten understanding of the root science involved.
Experts know what the general public knows, plus knowledge and experience. An infinite number of experts but who still only have publicly available information should not increase your confidence over polling a single median unbiased expert. (All the polling of n experts does for you is smooth out biases and errors)
A single piece of strong evidence would trump all experts, such as :
https://www.google.com/amp/s/www.forbes.com/sites/ericmack/2023/06/15/covid-19-patients-zero-in-wuhan-identified-boosting-lab-leak-theory/amp/
I have no way of knowing if the above rumor is true, just if the lab staff were patient 0 that is a smoking gun for the lab leak theory.
I would be curious about how valuable people think knowing the origins would really move the dial on predicting or preventing some future event.
I would think we have relatively good sample size for both zoonotic and lab-leak origins. Was there really something special about SARS-Cov-2 that getting the answer would really move our knowledge forward? If not, is it possible that the marginal step forward we might get is too little for all the political aspects that have existed?
I think there’s some controversy about which previous pandemics were caused by lab leaks (e.g. one flu season was speculated to have been), making the base rates less informative than you’d think.
Also I have just been made aware that only 22% of the experts claim to be aware of the DEFUSE grant proposal, versus 33% who claim to be aware of the nonexistent/fake Hanlen et al, 2022 study.
I think it’s reasonable to just completely disregard the opinions of experts who haven’t even heard of DEFUSE as it is such a crucial piece of evidence. Are these just bored virologists who read a few NYT articles and repeated the orthodoxy back? How could these people claim to be informed experts and not have heard of DEFUSE?
https://gcrinstitute.org/papers/069a_covid-origin-annex.pdf
Did you scroll down to see what people who were familiar with DEFUSE said?
The thing is, given that 33% of the respondents were “familiar” with Hanlen et al, 2022, it is hard for me to know how to interpret this.
What exactly are these 22%? Are they bored virologists who check boxes at random? Did they really thoroughly read DEFUSE and then it had absolutely no impact on them, so they’re like some kind of zombie?
I think the conclusion to this is basically to completely disregard expert opinion: listen to arguments, not to credentials.
Does it matter? The more important point here is that both zoonotic virus jumps and lab leaks are at-large risks that humanity should seek to reduce!
I think it does matter: I don’t think we have a strong sense of the base rate of lab leaks causing pandemics, so one sample would be informative.
Also, some of the experts would shift their views if the origin were known:
I hear one of the stated reasons for the labs is to study viruses and predict zoonotic jumps. At least some people think we were able to handle COVID so effectively because we were studying viruses in labs and anticipating what might happen, i.e. the net effect of labs is positive.
Given its size, it seems like whether COVID is in the ‘pro’ or ‘con’ column does a lot to our sense of whether or not this sort of virology has been good for humans or not and should continue into the future.
Another Insanity Wolf meme!
PREDICTS DISASTER
BY MAKING DISASTER
Did we need to know anything but “Covid is an airborne infectious respiratory virus”? How much research prior to the event did it take to know that?
On the one hand, yes, I agree; I thought virology research was crazy back in 2017? when someone at Event Horizon shared a paper which did a cost-benefit analysis and thought the net effect of BSL-4 labs was something like a hundred deaths per year per lab.
But I think it is important to be able to accurately understand what other people think so that you can talk to them instead of past them. (I still remember, with some bitterness, an op-ed exchange where two people debating virology said, roughly, “these things are so dangerous we shouldn’t study them” and “these things are so dangerous we have to study them”, and that was the end of the discussion, with agreement on the danger and no real ability to estimate the counterfactuals.)
This account of vaccine development claims that having done research on spike proteins back in 2016 was helpful in being able to rapidly develop the vaccine once the genome was uploaded, for example.
[To be clear, I think it’s important to distinguish here between gain of function research, which was disliked enough for there to be a funding moratorium (that then expired), and storing / working with dangerous viruses at all, which I think also is below the cost-benefit threshold, but this is a harder case to make.]
The virologists did not consider the topic about whether or not coronaviruses are airborne worth studying. They were rather assuming that it isn’t airborne and doing their research under safety protocols that don’t protect against airborne transmission.
If you actually want to know those things, funding virologists is useless and you instead want to fund epidemiologists that study disease transmission.
Surprised to read these threads without any reference to the Defuse proposal, documents of which were recently revealed to include even more detailed descriptions of features found in COVID-19 than the ones known previously.
Another thing that continues to surprise me in this discourse is not to have seen spelled out in terms of Bayes factors what to me seems the most straightforward way of thinking about the significance of the Defuse proposal as evidence, namely, that instead of thinking about the likelihoods of the features of the virus, we ask the competing hypotheses to explain the existence of the proposal itself and try to assess the Bayes factor P(Defuse|lab)/P(Defuse|natural).
Specifically, this involves estimating the probability that there exists a grant proposal, written less than 2 years before the outbreak by collaborators of the lab located in the city where the outbreak was first detected, proposing (among other things) to engineer a feature found in the natural virus:
not seen before in closely related viruses
defined by a 12 nucleotide insert when compared with closest relatives (i.e. the closest know relatives were not in any way close to evolving such a feature)
in the same location as the feature appears in the natural virus in the spike protein at the S1/S2 junction
These are not the only detailed correspondences between the supposedly natural virus and the grant proposal, as is made clear in the US right to know piece. To me, the existence of such a plan seems to be a fluke of cosmic proportions under the natural hypothesis and totally unsurprising under a lab engineering hypothesis.
The typical likelihood based argument I have seen argues (correctly in my view) that the likelihood|lab of certain features mentioned in Defuse is vastly elevated when conditioning on Defuse and that therefore P(feature|lab, Defuse) >>P(feature|natural). One thing about this form of reasoning that many people don’t seem to get is that the argument for P(feature|natural) being small is not based on some general view of what evolution is capable of achieving; instead it is based on the specificity of the feature in question. With Defuse, the challenge for the natural origin hypothesis is to argue for an elevated probability of a very specific outcome under the natural origin hypothesis. This can’t be done by appealing to the general awesomeness of evolution, as this form of argument only underscores the vastness of the sample space of natural viruses of which the subset of the viruses with the specific feature is a tiny subset.
What becomes clearer if instead of P(feature|lab, Defuse) we use up the Defuse evidence by thinking about P(Defuse|Hypothesis) is that the lab engineered virus hypothesis is the only hypothesis H one can think of under which P(Defuse|H) is not tiny.
I don’t doubt that a lab leak is possible, but I would be curious to hear arguments about how, if it was leaked from the lab in Wuhan, it was spreading in Italy and Brazil months earlier: https://www.sciencedirect.com/science/article/pii/S0013935122013068
It does make sense to me that a lab working with similar pathogens would be on the lookout for clusters of new disease and would be likely to sequence any new disease clusters.
“more likely caused by a lab accident (aka lab leak) or zoonotic spillover”
False dichotomy.
One thing you can be sure of in Establishment “debate”: the truth is not among the proffered options.
This video from 2019 of a Wuhan lab tech taking off his mask in a cave filled with bats certainly caused me to update my priors:
Relevant time stamp is about 4:20 to 5:00. The no-mask shots have the vague look of a filmic ‘re-enactment’ to me, also does anyone know if this video is from before or after the pandemic?