An alarm bell for the next pandemic
See my new post, “An alarm bell for the next pandemic, V. 2,” for updated criteria and more historical investigation.
My last post analyzed whether the sheer volume of COVID-19 posts on LessWrong could have alerted an average reader to the seriousness of the coming pandemic. I concluded that the answer was “no.” Other posters pointed out that some people did bring it up early, and that we still did an above-average job at acknowledging and preparing for the pandemic. But I am still looking for a reliable alarm bell that will ring loud enough to hear it before the stock market crashes 20% next time there’s a pandemic on the horizon.
This is my attempt.
Note: This questionnaire has passed two of my historical tests so far.
1) On 2009 H1N1 (swine flu) it performed correctly, not “ringing” in the context of a stock market that just went up all year long.
Trigger to investigate
Some threshold for investigating whether to ring the alarm bell is necessary. We can’t be constantly scanning the atmosphere for any sign of smoke to signal the next fire. So I want a trigger for when we should bother to look into the threat of pandemic more closely.
Most of us at least casually browse the news. Making the trigger news-linked seems sensible.
The first COVID article in the New York Times was from Jan. 8th, “China Identifies New Virus Causing Pneumonialike Illness.” It was on page A13. Several stories were published in the ensuing weeks. The first article on page A1 was “The Test a Deadly Coronavirus Outbreak Poses to China’s Leadership,” on January 21st, a month before the stock market crashed.
A headline article in a major newspaper seems like a good candidate as a trigger to investigate using the method I’ll explain below.
A simple model of pandemic risk
I created a four-factor model to evaluate pandemic risk. The factors are transmissibility, harm, spread, and institutional signals. Each are divided into several subcategories and are presented as questions. I believe each category could have been answered in a strong “yes” days or weeks prior to Feb. 20th, the edge of the stock market crash. Anyone following the news or capable of some quick scientific literature searches should have been able to answer them correctly without having a deeper understanding of the science or doing any mathematical modeling themselves.
Factor 1: Transmissibility
Does the disease appear to spread from human to human?
Does the disease spread via indirect contact (coughing, sneezing)?
Is there any evidence that the disease is transmissible before or just after the start of symptoms?
Do any academic papers, especially in the Lancet or the New England Journal of Medicine, suggest the possibility of “risk of much wider spread,” “exponential growth,” or use similar phrases? Try searching “[DISEASE NAME] exponential growth” on Google Scholar.
Factor 2: Harm
Is there evidence that the disease has a 3% or higher mortality rate among those 60 (or even younger), or 1%+ mortality among those 50 (or younger)?
Can the disease last for two weeks in more serious cases?
Do around 5% of patients seem to require hospitalization?
Are there no vaccines or treatments that have been proven effective?
Factor 3: Spread
Is the world death toll over 2,000?
Has the disease been detected in at least 10 industrialized countries collectively containing a total of at least 1 billion people, found in people with no clear link to the original source?
Is the disease present in major cities with strong international travel links?
Has the disease spread in advance of a lockdown, or escaped it?
Factor 4: Institutional response
Has a city-wide lockdown been attempted in a city of a population of at least 10 million, or have travel restrictions to or from major economies been implemented?
Has the WHO or a similar organization declared an “emergency of international concern” or “global emergency,” or issued an even more severe warning?
Have there been several front-page news stories about the disease?
Are there reports or warnings of shortages of medical supplies from the most-affected regions?
There are 16 questions here. My guess is that if you can answer 13 of them with a “yes,” it’s time to ring the alarm bell. Although it would be ideal if they all came with clear sources, it’s probably better to post and reconstruct your research later, than not to post at all.
Note: as Bucky pointed out in the comments below, it would be valuable to craft a set of “alarm bell” questionnaires which, based on the actual historical availability of COVID-19 data, would have been “ringing” a month, two weeks, one week, and two days before the stock market crash.
The cutoffs I chose here were all selected to have been passed prior to Feb. 20th, but for the death toll cutoff I just chose 2,000 as a nice round number, though we only hit that number a few days before the crash. I tried to strike a balance by requiring only 13 of 16 criteria, but that again was an arbitrary choice.
Deeper reasoning on these choices is of course ideal, but I think it’s still a reasonable starting point.
Ringing the alarm bell
What should you do if you run this checklist and get a score of at least 13?
Take a look at what Scott Aaronson wishes he’d told a friend on Feb. 4th. This is what he actually said, and what will likely seem reasonable to you next time you’re in the same situation:
For now, I think the risk from the ordinary flu is much much greater! But worth watching to see if it becomes a real pandemic.
The outcome for his friend?
A couple days ago, the same friend who emailed me on February 4, emailed again to tell me that both of her parents (who live outside the US) now have covid-19. Her father had to go to the emergency room and tested positive. Her mother stayed home with somewhat milder symptoms. Given the overloaded medical system in their country, neither can expect a high standard of care.
This is the start of how, in hindsight, he wishes he’d responded:
HOLY SHIT!!!!!—GET YOUR PARENTS SOMEWHERE SAFE—CANCEL ALL TRAVEL PLANS—STOCK UP ON FOOD AND MASKS AND HAND SANITIZERS. SELL ALL STOCK YOU OWN!!! SHORT THE MARKET IF YOU KNOW HOW, OTHERWISE GET CASH AND BONDS. HAVE AN ISOLATED PLACE TO ESCAPE TO. IF YOU’RE FEELING ALTRUISTIC, JOIN GROUPS MAKING THEIR OWN MASKS AND VENTILATORS.
Why didn’t he, or the rest of us, respond with this kind of tone and early timing?
We were all waiting for a gigantic, powerful, bright-yellow steamroller of a consensus to flatten our doubts. We needed the stock market to crash before we started really talking about it every single day. We needed everybody else to be talking about it first. We should not wait that long. We can’t afford to.
This is human nature, and while it’s embarrassing, it’s nothing to feel guilty or ashamed about. This community has a strong focus on preparing for future catastrophes and advocating for better preparedness. We should be learning from this now.
I think we need some kind of community standard for when we ring the alarm bell. There should be a standard for when anybody can do that, with no fear of embarrassment and a great deal of praise as long as they have justification.
In some factories, anybody can push the button to cease all work on the factory floor, for any reason, with no consequences if it was a false alarm.
This is probably too easy for our purposes. On the factory floor, you can see the issue with your own eyes. Your reputation with the colleagues you work alongside is on the line, regardless of the official consequences.
That’s why I like the idea of a checklist, and I’m submitting the one above as a tentative draft.
Responding to the alarm bell
If somebody rings the alarm bell, what should you do?
1. Because the first post may not have been based on the most accurate and up-to-date information, others should vet any sources submitted by the original poster, and look for better sources where they can. Compiling these sources, clipping out direct quotes that support each of the factors in the “should I ring the alarm bell” questionnaire, and writing up articles for easy sharing for a variety of audiences is a good step.
2. If my conjecture is correct, attaining a certain volume of new posts daily is key for making discussion and action on the topic feel normal. Early commentators and posters should focus on generating new topics for discussion and try to get new posts up quickly. The key is to create an appearance of robust discussion, in order to normalize it for people who are still hesitant to acknowledge pandemic as a real possibility and act accordingly.
3. Make it real for yourself. Stock up on a week’s worth of groceries, other necessities, nice-to-haves.
4. Decide on and publicly announce your threshold for selling stocks. The point isn’t to make a buck. It’s to represent the gravity of the situation in a real way, and pull probably the biggest lever you have to signal your prediction to the economy. Remember, when you sell stock, you’re not gambling: you’re sending a price signal to the world. That is you ringing the alarm bell.
5. Start a conversation with your friends and family. Post on social media. Write an op-ed. Ask trusted people to investigate the evidence for themselves and write about it so that the responsibility isn’t all on you. Remember, you’re trying to put the key in the ignition of that gigantic steamroller of consensus opinion and turn on the motor. Make memes.
But next time it will be different
It seems possible that we’ll see a global scale-up of stockpiles in the wake of this disaster. We’ll have accumulated a lot of cultural knowledge about how to respond to a pandemic. Pandemic will be available in our collective memory as a real possibility. We might put in place some strong preventative measures, such as a crackdown on wild animal markets. Policies will change. Technology will advance, giving us better ability to predict, treat, and vaccinate against the next illness. The market might be on a hair trigger next time around.
Ideally, we’d have periodic global review of the state of readiness.
A convenient report would be issued, perhaps in book form, giving a priori estimates of where we’d expect the next pandemic to originate from, how our infrastructure has improved since last time, how markets have responded to warning signs that we encounter in the coming decades. We’d have the information we need to judge whether things really are different.
Ideally, next time, the checklist I’ve created here will not predict the onset of mass global response to pandemic accurately. That could be because we’ve gotten so good at responding to pandemics that the same conditions don’t create cause for alarm. Or it could be that we’re so good at predicting them that the market responds much earlier, hopefully with the result that we never get to the same terrible state of affairs before government and NGOs are motivated to act. Either the building gets fireproofed or the alarm bell gets more sensitive.
Modeling whether the situation has changed
It would be valuable to construct some models akin to the one above to determine whether the context for pandemic risk fundamentally changes in the future.
1. List the major weak points of our current pandemic infrastructure. Can we point to ways in which they’ve substantially improved compared to where we were pre-COVID-19?
2. Look at the organizations involved in signalling the next pandemic: the WHO, the CDC and its counterparts in other nations, the popular press, the Bill and Melinda Gates Foundation (who donated $100 million to fight COVID-19 in January). What would we look for to gauge their threshold for ringing the alarm bell? How would we know if it has changed relative to 2020?
3. Our improving synthetic biology infrastructure will give us more resources to monitor, treat and prevent disease. It will also give us the power to make deadlier diseases. What technological improvements would expand the ability of a rogue actor either to resurrect an old superbug like smallpox, or to create a new one? What real security do we have against this possibility? With time, what is expanding more quickly: our ability to prevent or cause harm?
4. Who nailed this pandemic prediction on the head, with evidence and arguments and confidence that make us feel like it wasn’t just a lucky guess? Is there a convenient way we can keep track of them so that we have a convenient source of guidance next time? We need to establish our go-to source of wisdom so that we don’t have to do all the thinking in the middle of the next crisis.
Conclusion
In a pandemic, a matter of weeks can make a difference of many thousands of lives, billions or trillions of dollars, and immense social disruption.
We will see another pandemic.
It might be worse. There’s no clear reason why mortality and transmissibility of a virus should be inversely correlated. The next virus could be both more catching and more fatal than COVID-19.
Having clues and some commentary and a few posts here and there isn’t enough.
Right now, we can’t trust the established authorities to nail this. Is anybody feeling hunky dory about the way world governments are behaving now, or how they will change over time?
Nature won’t provide us with a grace period. Rolling a 1 on a 20-sided die doesn’t make the odds of another 1 on your next roll any less likely. Getting a heads on a coin flip doesn’t increase the likelihood of getting a tails next time. The next pandemic could happen any time.
More quickly fatal diseases leave less time for the immune system to respond and less time for transmission to occur. You’re right that’s not to say we can’t end up with diseases that are both more contagious and more deadly than COVID-19, we definitely could, but that’s not the direction the correlation goes.
That’s a good point. It matters more for diseases where the window of transmissibility and of symptoms closely overlap. The contagiousness of the disease is another critical factor that isn’t affected by how quickly fatal the disease is. Overall, I think my statement is right, but you supply valuable nuance.
I used to browse flutrackers.com (it is like LessWrong for pandemic risks) and I knew about the new infection from day one (early January). If you want to have an alarm bell about the next pandemic, check it often.
More generally: If one wants to have an alarm bell for a problem X, he should find a forum of geeks who write about X and check it regularly.
My main failure was that I didn’t write about the coronavirus earlier in LW (before January 20) and that my first post title “Rational prepper thread” didn’t have the word “coronavirus” in it.
It would be nice if we had somebody who specifically agreed to check sites like that regularly and then let the rest of us know if there’s something worth being concerned about. That way we could avoid duplication of effort.
I know I’m a bit late, but that sounds like something that could (up to a point) be reasonably automated.
Apart from flutrackers.com, are there other sites in a similar trend that should be bell-checked? has there been a previous effort to create a general alarm bell along the lines of these ideas?
My “threshold” criteria here should help. It seems we’ll likely get at least one front-page headline about a novel illness, and more likely many headlines, before the world reacts. My guess is that for those of us not working professionally in pandemic prevention, we’ll know early enough from the press to be on watch. The point of this model is to get us to the next step, which is reasoning under uncertainty and acting appropriately.
By the way, check out my updated alarm bell criteria, which I just posted yesterday.
The thing I regret the most was not explicitly debunking myths that were common back in the early days. For example, a lot of people told me that this was just going to disappear like SARS. But I actually had got my hands on a dataset and SARS looked way different. There was little reason to believe it was like SARS.
I also didn’t give people financial advice because I’m very averse to doing stuff like that, and I thought people would generally just say that I’m “trying to beat the market” against a common consensus that beating the market was impossible. Though once stuff started happening, I started to try my hand at it.
I started having panic attacks and needing to find creative places to store my accumulated nonperishable supplies shortly after Valentine’s day.
My attempts to warn people were primarily directed towards my university, my family and those I love. My boss had us ready for cessation of in person activity two weeks before others as a result and he may have helped convince a county to close schools earlier than otherwise. Family hasn’t had to go to a grocery store for 5 weeks.
This place was a bit far from my mind due to drifting away, unfortunately.
There will be a lot more of these this century. What other possible outcome could there be to becoming 40% of mammalian biomass on the planet, with such a well-mixed system?
Indeed, I don’t know why ‘once a century’ would be expected to hold with record amounts of density, travel, and absolute numbers of people in unsanitary conditions.
Is one correct inference to take here that such viral mutations are that common but previously emerged and then disappeared without widespread human infection due to the natural isolation and separation that exists for several reasons?
There’s the fact that most viral spillovers into humans don’t actually transmit well, and the fact that if they do but burn out it doesn’t really matter on the large scale. The latter has definitely changed.
HIV is an interesting case study. Genetically, we can tell it leaped into humans from great apes TWICE in the 1920s (HIV-1 and HIV-2) in a way that kept transmitting around the globe. This probably had to do with the urbanization of sub-Saharan Africa—the sudden sucking of huge numbers of rural people with contact with bushmeat into globally connected urban centers. Presumably it had been transmitting into people forever, just always burning out.
A few years back there was an antibody study in a village near a bat cave in China that found that 0.5% of the people that lived a few hundred meters from the cave showed antibodies against SARS-like viruses.
I suspect that this will incite governments to care a lot more about any potential pandemics in the future, since no one is going to forget this for the next twenty years. If anything, I’d put extra resources into other major dangers that aren’t pandemics.
We’ll have to wait and see. That’s why I want a model for how to compare our anti-pandemic infrastructure at intervals over the next 20 years.
You looked at whether your criteria where reasonable for the crisis this year but that’s not enough to tell whether the criteria where positive in this crisis but also how often they were positive in the last two decades.
Without engaging in deep research myself I think there’s a good chance that H1N1 triggered the criteria you listed but it would have been a mistake to sell stocks in response.
I decided to make H1N1 the first alarm bell test. It performed correctly, not ringing!
That’s great, and does give me more confidence in the alarm bell being useful.
Are you talking about Spanish flu in 1918 (which was caused by H1N1), or a more recent outbreak?
I did mean the 2009 outbreak.
From context it’s presumably the 2009 outbreak where roughly ~60 million got infected in the US
Did you estimate how early using this would have caused an alarm to be raised for COVID-19?
I think the top 3 the harm questions were confirmed in this paper on 11th Feb but maybe there were other papers before this or we could have inferred from public data?
2,000 deaths was 18th Feb.
Escaping a lockdown attempt would probably be ~21st Feb in South Korea (the virus didn’t really escape China lockdown—it had escaped before the lockdown)
Indirect transmissibility I’m not quite sure about a date?
Pre-symptomatic transmission again I’m not sure—from the papers in jimrandomh’s post maybe early-mid Feb we had a good hint.
These are good questions. My question re: the lockdown probably needs to be worded differently, because I meant it as “a lockdown has been attempted at the point of origin and yet the virus continued to spread,” which we knew had happened once the virus showed up in Thailand.
I knew this would only have given early warning a few days before the stock market crash due to the deaths cutoff. I wanted a fairly conservative alarm bell. But maybe a good refinement would be to create a graduated series of alarm bell questionnaires based on the same criteria, but with cutoffs that would have caused it to “ring” a month, two weeks, one week, and two days before the stock market crash.
I’ll add this idea to the OP and also reword the “escape lockdown” criteria. Thanks for the feedback!
I think even a few days has the potential to be extremely valuable if it can be pulled off. If worldwide reactions had happened a few days sooner then half of the cases could have been avoided. LW ringing an alarm bell a few days earlier might not have had an effect on policy but its important to note how big the potential gains are.
As you say in the OP, the next time any pandemic comes along the worldwide response is likely to be better. So my main question is how do we generalise this advice for other severe dangers.
To me one of the main issues if the speed at which things happen. Most things which happen gradually give enough time for people to react without disastrous consequences—COVID only gives a few days before your problem is doubled. This would be fairly high on my checklist specifically for a future pandemic—low doubling times—but for general alarm bell ringing speed of problem development should also be up there.
*insert obligatory FOOM comment here...*
When it comes to optimizing the alarm bells it would be good to have a large Google Doc with different crisis, different criteria and then see which criteria are best for early warnings.