FHI paper on COVID-19 government countermeasures

Link post

In brief: this is the largest data-driven study try­ing to dis­en­tan­gle the effect of in­di­vi­d­ual coun­ter­mea­sures, and one of the most thor­oughly val­i­dated ones. The au­thors ran mul­ti­ple pre­dic­tions with held­out data and var­i­ous sen­si­tivity analy­ses. The re­sults seem ro­bust. It’s still un­clear how gen­er­al­is­able they are, since it’s es­sen­tially an ob­ser­va­tional study. (I man­aged the pro­ject who wrote the pa­per, but I wasn’t in­volved as a co-au­thor.)

The effec­tive­ness and per­ceived bur­den of non­phar­ma­ceu­ti­cal in­ter­ven­tions against COVID-19 trans­mis­sion: a mod­el­ling study with 41 countries

Back­ground: Ex­ist­ing analy­ses of non­phar­ma­ceu­ti­cal in­ter­ven­tions (NPIs) against COVID19 trans­mis­sion have con­cen­trated on the joint effec­tive­ness of large-scale NPIs. With in­creas­ing data, we can move be­yond es­ti­mat­ing joint effects to­wards dis­en­tan­gling in­di­vi­d­ual effects. In ad­di­tion to effec­tive­ness, policy de­ci­sions ought to ac­count for the bur­den placed by differ­ent NPIs on the pop­u­la­tion.

Meth­ods: To our knowl­edge, this is the largest data-driven study of NPI effec­tive­ness to date. We col­lected chronolog­i­cal data on 9 NPIs in 41 coun­tries be­tween Jan­uary and April 2020, us­ing ex­ten­sive fact-check­ing to en­sure high data qual­ity. We in­fer NPI effec­tive­ness with a novel semi-mechanis­tic Bayesian hi­er­ar­chi­cal model, mod­el­ling both con­firmed cases and deaths to in­crease the sig­nal from which NPI effects can be in­ferred. Fi­nally, we study how much per­ceived bur­den differ­ent NPIs im­pose on the pop­u­la­tion with an on­line sur­vey of prefer­ences us­ing the MaxDiff method.

Re­sults: Eight NPIs have a >95% pos­te­rior prob­a­bil­ity of be­ing effec­tive: clos­ing schools (mean re­duc­tion in R: 50%; 95% cred­ible in­ter­val: 39%-59%), clos­ing nonessen­tial busi­nesses (34%; 16%-49%), clos­ing high-risk busi­nesses (26%; 8%-42%), and limit­ing gath­er­ings to 10 peo­ple or less (28%; 8%-45%), to 100 peo­ple or less (17%; −3%-35%), to 1000 peo­ple or less (16%; −2%-31%), is­su­ing stay-at-home or­ders (14%; −2%-29%), and test­ing pa­tients with res­pi­ra­tory symp­toms (13%; −1%-26%). As val­i­da­tion is cru­cial for NPI mod­els, we performed 15 sen­si­tivity analy­ses and eval­u­ated pre­dic­tions on un­seen data, find­ing strong sup­port for our re­sults. We com­bine the effec­tive­ness and prefer­ence re­sults to es­ti­mate effec­tive­ness-to-bur­den ra­tios.

Con­clu­sions: Our re­sults sug­gest a sur­pris­ingly large role for schools in COVID-19 trans­mis­sion, a con­tri­bu­tion to the on­go­ing de­bate about the rele­vance of asymp­tomatic car­ri­ers in dis­ease spread­ing. We iden­tify ad­di­tional in­ter­ven­tions with good effec­tive­ness-bur­den trade­offs, namely symp­tomatic test­ing, clos­ing high-risk busi­nesses, and limit­ing gath­er­ing size. Clos­ing most nonessen­tial busi­nesses and is­su­ing stay-at-home or­ders im­pose a high bur­den while hav­ing a limited ad­di­tional effect.

The team who pro­duced this work is also hiring for a new pro­ject man­ager.