Flaw #1: The model assumes the measures most states are taking now will be as effective as the lockdown in Wuhan
The model assumes, without evidence or justification, that if we take 3 of the following 4 measures (school closings, closing non-essential services, shelter-in-place order, major travel restrictions) it will “be enough to follow a trajectory similar to Wuhan”. Many newspaper articles have covered the extreme measures enacted in Wuhan, but I will summarize them here:
[...]
Flaw 3: The model chooses not to incorporate an increased mortality rate for Covid19 patients unable to receive medical care due to overwhelm of the health care system.
I was suspicious of the IHME model several days ago when I first saw it, but couldn’t find a detailed description of their methodology. (It’s really well hidden, and doesn’t even appear in their FAQ section.) Finally found it yesterday, noticed the “similar to Wuhan” assumption, then saw the page criticizing it linked in the comments section for the paper.
Can anyone help me to understand these graphs? The ‘deaths per day’ graphs seem to incorporate actual data up to April 1st and to be projections from there, which suggests that they are using the actual death rates to calibrate some parameters of their model. But the ‘hospitalization’ graphs don’t seem to incorporate any actual data, and seem to be very different from the reality: for example, for April 1st (when the models were last updated), the “all beds needed” and “ICU beds needed” numbers for New York State are 50,962 and 10,050; but it looks like the actual number of people in hospital for Covid in NYS on that date was about 12,500, with about 3,000 in the ICU (source: ‘NYS total hospitalized’ graphs here [https://gothamist.com/news/coronavirus-statistics-tracking-epidemic-new-york]). Does that mean that they are not trying to fit their model to actual hospitalization data at all? If so that seems like a problem: hospitalization rates should be much better than death rates for estimating what effect social distancing measures are having, since there is so much time between infection and death.
IHME published a dashboard with state-by-state projections of coronavirus peaks: http://covid19.healthdata.org/projections
The accompanying FAQ is also interesting: http://www.healthdata.org/covid/faqs
The IHME Covid19 Model is Dangerously Flawed:
Thanks, hadn’t seen that.
Also just saw this, which makes a lot of the same points: https://westhunt.wordpress.com/2020/04/04/ihme-projections/
I was suspicious of the IHME model several days ago when I first saw it, but couldn’t find a detailed description of their methodology. (It’s really well hidden, and doesn’t even appear in their FAQ section.) Finally found it yesterday, noticed the “similar to Wuhan” assumption, then saw the page criticizing it linked in the comments section for the paper.
Can anyone help me to understand these graphs? The ‘deaths per day’ graphs seem to incorporate actual data up to April 1st and to be projections from there, which suggests that they are using the actual death rates to calibrate some parameters of their model. But the ‘hospitalization’ graphs don’t seem to incorporate any actual data, and seem to be very different from the reality: for example, for April 1st (when the models were last updated), the “all beds needed” and “ICU beds needed” numbers for New York State are 50,962 and 10,050; but it looks like the actual number of people in hospital for Covid in NYS on that date was about 12,500, with about 3,000 in the ICU (source: ‘NYS total hospitalized’ graphs here [https://gothamist.com/news/coronavirus-statistics-tracking-epidemic-new-york]). Does that mean that they are not trying to fit their model to actual hospitalization data at all? If so that seems like a problem: hospitalization rates should be much better than death rates for estimating what effect social distancing measures are having, since there is so much time between infection and death.