In terms of true number of infected, I’m predicting that SK has on the order of 100K to 200K cases and say 4K in Iceland, and I don’t find this up to ~50x difference very surprising. Firstly, it’s only about an 18 day difference in terms of first seed case at 25% daily growth.
I see. BTW our confidence intervals for the IFR have some overlap: 0.4% is my lower bound and your higher bound. :)
Edit: changed some numbers slightly after looking things up, to make them more accurate.
Modeling one country as “X weeks behind” some other country is hazardous at best and also unnecessary as Iceland provides direct graphs on their daily #tests and #positive.
I agree that it’s tricky to do the modelling correctly, but I feel like you’re not engaging with my point properly. I think the following argument I made is watertight:
There was a point when South Korea had several deaths (50ish) and thousands of cases (7,700) and their IFR was at 0.6%.
That’s roughly when they got their outbreak under control. The numbers slowed down tremendously, and 20 days later they are only at 9,500.
So in those 20 days, the reported CFR 2.5xed.
Iceland’s reported CFR never 2.5xed so far.
Therefore, they are way behind South Korea’s timeline even if we grant the point that Iceland has it’s outbreak contained (you may be completely right about this, because I didn’t follow it).
The way I see it, this point is only wrong if somehow Iceland going from 1 deaths to 2 deaths is the equivalent stage of the timeline as South Korea’s deaths going from 50 to 144 (or whatever the numbers were). That seems highly improbable to me because it would mean that South Korea’s outbreak was 50 times larger than Iceland’s. That doesn’t seem right to me. (Though I guess if I had a strong belief that the hypothesis you’re defending is consistent with other data points, then this may not be a knockdown argument by itself? Would you expect South Korea’s outbreak to be 50x larger? No need to answer, of course. But if this argument updates you somehow, I’d be curious to hear!)
This person has been collecting reports in Italian media and also contact the mayors of Italian district to request the information.
It’s not for all of Northern Italy, but it’s also not villages either. Those cities or provinces are much more populated than I initially thought (see Stefan Schubert’s correction to my comment elsewhere).
Additionally, if most cases were asymptomatic or weakly symptomatic there would be few cases of multiple close contacts becoming ill. These are common.
I think this point is really underappreciated.
Minor point because I agree with all the other things you said: While it’s true that South Korea didn’t have a China-style lockdown, I think the behavioral changes at the city level must have been really quite extreme. Perhaps in part also culture-driven rather than government imposed (maybe South Koreans actually followed government recommendations almost perfectly?), but I think it could be overupdating on the evidence to assume that South Korea didn’t need some kind of “lockdown” (loosely spoken) to get the situation under control initially. I’m not 100% sure this is what happened, but I heard at least one expert say that people who claim South Korea didn’t have a lockdown are being misleading, and that point also seems to make sense based on priors.
So the testing of several hundred thousand cult members pushed both their CFR and test positive fraction lower than it otherwise would be, and rather obviously skewed their case age structure.
It skewed the age structure toward a younger demographic. Were you aware of this or did you assume that the religious group is skewed toward old people like typical churches? I didn’t realize this up until like ten days ago, but the Christian cult was predominantly pretty young people!
Nonetheless they have tested far less of their population than Iceland (about 5X less as of 3⁄20 according to ourworldindata), so if the ratio of infections/cases is 4x to 5x in Iceland it seems reasonable that it’s 10x to 20x in SK.
The reason I don’t consider it at all plausible that South Korea missed 80% or more of its cases is because of how quickly and lastingly they were able to gain control over their outbreak.
And about Iceland: Isn’t it really very clear that Iceland is weeks behind South Korea, and that Iceland’s numbers are therefore unrepresentatively low? For comparison, South Korea’s IFR was 0.6% at a point when they had 7,700 confirmed cases. I think this was roughly 20 days ago. So 20 days for South Korea’s IFR to go from 0.6% to 1.5% is how long it takes a majority of patients to die if the hospital conditions are favorable enough to give everyone good treatment. There weren’t many new confirmed cases in the meantime because the current count is 9,500. So with respect to the IFR Iceland is currently at (0.21%), if Iceland had their outbreak under control, we should expect that IFR to rise by a factor >2.5x. 2.5x is the lower bound because South Korea’s IFR was at 0.6% at a time when they already had dozens of deaths; by contrast, Iceland only has two deaths so they are way behind the timeline. (This comes from the effect that once true cases stop growing, the CFR rises up until all the illnesses take their course.) Expecting anything lower than a 4x increase from time delay is unreasonably low. So to make Iceland’s reported CFR comparable to South Korea’s, we should think of it as 0.84% rather than 0.21%. And then we can think about how many cases went undiagnosed in both countries (but maybe you did factor this in).
In addition, we have to factor in that Iceland doesn’t have their outbreak under control. Or do they? I didn’t check up on this, but I’d be surprised if they had the outbreak contained. My guess is they caught fewer cases than South Korea! Yes, Iceland did more testing per capita. But South Korea knew where to look! They really managed to get their outbreak under control. It’s very impressive and I feel like they’re not getting the credit they deserve.
Anyway, assuming Iceland still has community transmission, this would mean that through new testing, new confirmed cases will be added constantly to the total. Those cases will predominantly be recently confirmed cases where not enough time had passed for people to die. This will keep Iceland’s reported CFR at a low level for quite a while to come, but this provides basically zero evidence for the actual IFR being low.
FWIW, if UK death toll will surpass 10,000, then this wouldn’t fit very well with this hypothesis here.
If this update works then I feel like just looking at how the numbers in Italy came together would change your mind about the low-IFR hypothesis.
Alternatively, if the Covid-19 deaths in NY state go above 3,333 in the first week of April, that seems like it would also falsify the hypothesis. (NY state has fewer than one third the population of the UK.) Unfortunately I think this is >80% to happen.
The “COVID-19 is similar to influenza” model predicts IFR in the 1% range for a retiree age distribution like on DP but 0.1% range on the US age distribution.
FWIW I tried to do an age adjustment for the Diamond Princess myself and what I got was that the 1.4% IFR for the cruise demographics translates into a 0.3% IFR for US demographics (factoring out gender adjustments). I think you could argue that because women were underrepresented on the cruise ship, the adjustment should be greater, so 0.25% is plausible. That said, this doesn’t yet factor in that the people who are medically worst off probably don’t book cruises, so my best-estimate adjustment is maybe 0.4% with a lot of uncertainty. I agree that the people who use the Diamond Princess as evidence for an IFR around 0.9% or higher seem to be making a mistake. At the same time, I do think the Diamond Princess is at least weak to moderate evidence against the 0.125% figure Ioannidis arrived at, or the 0.1% figure that I’ve seen discussed elsewhere.
I don’t really know how this compares to flu mortality, but I found myself somewhat skeptical about the claim I quoted above. You seem to get a 10x update for your age adjustment, whereas my update was only about 5.7x (before factoring in harder-to-quantify assumptions that IMO reduce the factor a bit more even).
(I made a huge mess of my calculations and I don’t recommend clicking on the following link, but just so people see that I’m not just making this up, here’s some evidence that I did something with numbers. Could also be that I neglected some considerations. For factoring in how much overrepresentation of age bracket 70-79 changes things, I based the adjustment off of previous estimates on how strongly Covid19′s IFR is age skewed. I’d imagine that this adjustment was uncontroversial because whether you subscribe to the low IFR theory or not, probably there’s no reason to question whether the proportionalities of the attack rate are correctly reported?)
This is a non-random village in Italy, so of course, some villages in Italy will show very high mortality just by chance.
It’s extremely implausible that it would be 10x or 15x higher than what’s expected for the typical Italian village. Besides, other villages like Cremona or Bergamo also seem to be close to those numbers. Smoking or age structure or air pollution doesn’t give you a 10x update.
UPDATE: Wow, I was totally wrong about those being villages. As Stefan Schubert pointed out, those are cities and provinces with tens and hundreds of thousands of inhabitants!
In Italy, with almost 10k deaths it would be 0.02%-0.04%
There’s an Italian village where 0.1% of the population already died with a confirmed diagnosis of Covid-19. Inferring from typical monthly death rates it’s also estimated that the twice as many people died from Covid-19 in that village without an official diagnosis. There’s a bunch of uncertainty about those additional 0.2%, but it would put the fatality rate at 0.3% already. And those figures are from 4 days ago (edit: 6 days ago actually).
Edit: It’s a province and city(!), not a village.
One interesting thing is that people in their 20s and 30s had a much higher rate of symptoms (80%+) than older or younger people (< 60%).
That’s indeed interesting. This article seems to say that it’s different in young people who tested positive in South Korea.
One point of criticism about the link included under “Diamond Princess data:”
The abstract reads as follows:
Comparing deaths onboard with expected deaths based on naive CFR estimates using China data, we estimate IFR and CFR in China to be 0.5% (95% CI: 0.2–1.2%) and 1.1% (95% CI: 0.3–2.4%) respectively.
This wording to me suggests that the authors think 0.5% is most likely the correct IFR for China’s cases (up to a certain point in time, based on when the paper was written). This is either false, or the authors are making a really obvious mistake. The paper did not factor in that sick people in Wuhan (where most of China’s cases were from at the time) probably had much worse treatment prospects than patients from the cruise ship.
Excellent analysis! I’ve used the 18% point estimate from that Diamond Princess study without noticing that the math could be off.
One point to add: I’ve seen people say that asymptomatic presentations of SARS-CoV-2 infection might more common in young people, especially in the age range from 20-40. That age range was underrepresented on the cruise ship. For that reason, perhaps it’s possible for up to 65% of cases to be asymptomatic?
That said, I very much agree with you that the entire thing about asymptomatic presentations could be a myth based on false positives and confusing “asymptomatic” with “pre-symptomatic.” This study is the type of thing that would give us confidence in the existence of asymptomatic carriers – if only it had more examples than just one person.
I read about the new deaths on the Wikipedia article.
A Canadian man in his 70s died on 19 March, making him the ninth coronavirus-related death from the ship. Two Japanese passengers in their 70s died on 22 March.
I know that. If you follow this discussion up to the beginning, you’ll see that all I’m claiming is that the number of documented cases has been affected by selective bias, because asymptomatic / pre-symptomatic etc. cases are unlikely to be diagnosed.
Okay. I feel like the discussion is sometimes a bit weird because the claim that there are a lot of undocumented cases is something that both sides (high IFR or low IFR) agree on. The question is how large that portion is. You’re right to point to some sampling biases and so on, but the article under discussion estimates an IFR that it at least a factor 5 below that of other studies, and a factor of 4 (or 3.5 respectively) below what I think are defensible lower bounds based on analysis of South Korea or the cruise ship. I don’t think selection bias can explain this (at least not on the cruise ship; I agree that the hypothesis works for China’s numbers but my point is that it conflicts with other things we know). (And I already tried to adjust for selection bias with my personal lower bounds.)
I’m not saying this study is right. I’m just saying that, unless someone points a methodological flaw, “their conclusion is too different” is not a reason to discard it.
It depends on the reasoning. We have three data sets (there are more, but those three are the ones I’m most familiar with):
The Diamond Princess
How much to count evidence from each data set depends on how much model uncertainty we have about the processes that generated the data, how fine-grained the reporting has been, and how large the sample sizes are. China is good on sample size but poor in every other respect. The cruise ship is poor on sample size but great in every other respect. South Korea is good in every respect.
If I get lower bounds of 0.4% and 0.35% from the first two examples, and someone writes a new paper on China (where model uncertainty is by far highest) and gets a conclusion that is 16x lower than some other reputable previous estimates (where BTW no one has pointed out a methodological flaw either so far), it doesn’t matter whether I can find a flaw in the study design or not. The conclusion is too implausible compared to the paucity of the data set that it’s from. It surely counts as some evidence and I’m inclined to move a bit closer to my lower bounds, all else equal, but for me it’s not enough to overthrow other things that I believe we already know.
They write “at the time of testing.” The study I cite followed up with what happened to patients.
Also relevant: In the last 5 days, 3 more people who had tested positive on the Diamond Princess died. And one person died two weeks ago but somehow it wasn’t reported for a while. So while my own estimates were based on the assumption that 7 / 700 people died, it’s now 11 / 700.
Older people are more likely to get infected, so the infected population in the US will lean older as well—closer to the distribution on the ship.
Interesting! Do you think this is established? I haven’t looked into this, but my guess would have been that the risk is similar because young people are less scared of the virus. But yeah, good point about further adjustments being needed to get the best estimate.
From the paper:
With testing capacities of 20,000 tests daily, it [South Korea] has the largest and most accurate coverage compared to all other countries as of writing. The low false-negative rate in detecting COVID-19 infections leads to the lowest death rate compared to all other countries (0.84) with major case count
Note that South Korea’s reported (naive) CFR is at >1% by now. It’s possible that the authors adjusted for the fact that most of South Korea’s cases were still active at the time of writing (about 55-60% of cases are still active now, I think), but I don’t see this in this paper. It probably doesn’t make a huge difference, but still relevant that this could cause the estimates to be a bit too low.
This method requires the comparison of two countries with sufficient confirmed cases and reported deaths. One country (target country) will be adjusted, given the information from the second country (benchmark country). In order to adjust for the difference in the population demographics of the target country, T, and the benchmark country, B, we compute a Vulnerability Factor (VTB).
Am I right that they’re not factoring in that patients had worse prospects in Wuhan than in South Korea? I feel like whatever the outcome of their adjustment process, that value would need to be multiplied by a factor >1 which represents hospital overstrain in Hubei, where at least 60% of China’s numbers stem from (probably more but I haven’t looked it up). I don’t know how large that adjustment should be exactly, but I find it weird that there’s no discussion of this. Am missing something about the methodology (maybe it factors in such differences automatically somehow)?
Ah, OK: They list this as an assumption:
[Assumption]Treatment has minor influence on outcome The provided healthcare in countries is comparable. For developed countries such as Italy and South Korea, it is assumed that the population has similar access to treatment. The death rates reported by age group are thus applicable in all countries
This is important to keep in mind when we try to derive implications from their estimate. Especially if we look at the hospitalization rates estimated here on page 5. For this disease in particular where people sometimes have to stay in hospitals for several weeks, it’s hard to imagine that treatment only makes a small difference.
More points in favor of a higher IFR:
The percentage of asymptomatic cases on the Diamond Princess was even lower than 50%. It was only about 18%. (I trust this figure because the paper has author overlap with the paper that gave a higher figure initially, and it’s written by the same author who made the 0.1% estimate and we’d expect this person to – if anything – have a bias toward expecting a larger number of asymptomatic cases).
About the age distribution on the Diamond Princess: I tried doing age adjustment for it here. ((Edited because I revised some estimates.))
I also thought that in Lombardia, the estimates given by Ioannidis are rapidly trending toward coming in contradiction with SIR models. :( Lombardia has a population of 11 million people and 2,500 reported deaths. Some doctors are raising alarm that many deaths are going undetected because people are dying at a rate that’s 4 times higher than the same month last year. In addition, the death counts always lags behind because some people are sick for a long time before they die (though maybe this start to be the case less strongly in conditions of extreme hospital overstrain). All of this suggests that an estimate of 10,000 deaths for Lombardia alone might soon prove to be accurate. But according to the IFR provided by Ioannidis, this would correspond to an expected 8 million people infected (72% of the population). I don’t understand SIR models well enough to calculate what the R0 would have to be for 72% of a population to get infected. I suspect that Covid-19′s R0 is high enough to be consistent with this, but it wouldn’t leave a lot of room for estimation errors.
That said, I think the above calculation is naive, so the argument doesn’t work (at least not in this crude form). If hospitals become as overwhelmed as they are in Italy, I’m sure that even someone with Ioannidis’ view would expect the IFR for Lombardy to become a lot higher than 0.125% because a lot of people aren’t getting life-saving hospital attention.
So, this means that Lombardy isn’t necessarily a knockdown argument against Ioannidis’s estimate in the same way South Korea is. However, I think Ioannidis’s estimate would have counterintuitive implications for the percentage of people infected in Lombardy. It would have to be in the double digits already at the very least. The most trustworthy estimate I saw about Wuhan suggested that only 5% of its population had the virus. However, there’s some disagreement about this, and the people who tend to argue for an unusually low IFR also tend to argue that there’s a giant iceberg of undetected asymptomatic cases.
UPDATE: I just realized something: I read somewhere recently that Italy is doing 30,000 tests a day by now, and that about 25% of them are positive. This seems to be in contradiction with Ioannidis’s estimate because his view should imply that, if there’s some kind of selection at all for who they are testing (as opposed to just testing members of the population at random), then we should expect to see more positive test results than 25%. (Why? Because if we assume that hospital overstrain increases his death rate estimate by a factor of 7x (which is a really large adjustment!), the death count estimates for Lombardy combined with Ioannidis’s estimates would still suggest that above 10% of the population would have the virus. Such high numbers would only be consistent with reality if most people had relatively mild symptoms or no symptoms at all, so assuming that there’s substantial pre-selection on who is getting tested (as opposed to random testing, which would be odd), a rate of 25% positive tests would be implausibly low for the scenario where >10% of the region were infected. So, to conclude, I think one can plausibly construct a case against Ioannidis’s estimates based solely on common sense and numbers from Lombardia. I probably haven’t quite succeeded at making this case in a watertight way, but I think you might be right with your intuition. This is just one more reason why the 0.125% estimate is completely absurd.
Reading the Ioannidis article, it seems to say that he did his own calculations, and he doesn’t show them. Okay.
I’m curious about this, so I’m going to try a ballparking estimate myself.
Tl;dr I intially arrived at a result that suggested 0.125% was way off, but then found better info on the cruise ship’s age distribution and had to revise my judgment. I now find it debatable whether 0.125% is defensible or not, but it’s not “way off.” My own estimate would be more in the ballpark of 0.3%, but I don’t anymore consider the cruise ship to be evidence for IFR estimates at 0.5% or higher.
Update March 24th: In the couple of days, 3 new patients who had tested positive on the Diamond Princess have died. In addition, the Wikipedia article has been edited to list another death that previously hadn’t been included. So total deaths per confirmed cases on the Diamond Princess are now 11 / 700 instead of 7 / 700. All my calculations below are based on the older, outdated numbers. To get the most updated estimates, just multiply the results below by 11⁄7.
Note that I have never done age adjustments for anything, so I have no idea what the proper methdology would be. I’m just curious to see if 0.125% is potentially reasonable rather than (as my current intuition suggests) very dubious.
From this paper, I found the following info:
A total of 634 people tested positive among 3,063 tests as at 20 February 2020. Of 634 cases, a total of 313 cases were female and six were aged 0–19 years, 152 were aged 20–59 years and 476 were 60 years and older.
At the end of the outbreak, roughly 700 people had tested positive. I’m going to assume that the 66 patients not yet in the above statistics fall into age categories in the same proportion. So a bit more than two thirds of the 66 patients get added to the 476 figure for people aged 60 and older.
With this adjustment, we have 700 diagnosed cases, of which an estimated 525 patients were aged 60 and older. Of those 700 diagnosed cases, 7 people died. 525 out of 700 corresponds to 75%. (I’m going to mostly ignore the death risk for people below age 60 for the analysis below, because it will be negligible given that people older than that anyway make up the majority share.)
This wikipedia article on US demographics says the following:
0–14 years: 18.62%
15–24 years: 13.12%
25–54 years: 39.29%
55–64 years: 12.94%
65 years and over: 16.03%
Eyeballing this, let’s go with 22% of the population at age 60 or older.
75 divided by 22 is roughly 3.4, so this naively suggests that the cruise ship’s demographic was roughly 3.4 times more susceptible to dying from SARS-CoV-2. If I divide the observed IFR of 1% by 3.4, I get 0.3%. Why does Ioannidis get 0.125% instead of 0.3?
Moreover, it seems to me that 0.3% must be an underestimate because I assume that even though the cruise ship population is substantially older on average than the US population, I would think that this effect will disappear (or even reverse) at the extremes, once we look at the percentage of exceptionally old people (e.g., aged 80 and above, age 85 and above, etc.). Because Covid-19 is particularly fatal for the very oldest people, I expect the 0.3% figure to contain a substantial degree of overcorrection. Especially also because elderly people with the most severe pre-existing health conditions are likely heavily underrepresented on cruise ships. This effect could be really quite significant: It’s not even totally obvious that a downward adjustment of the 1% IFR observed on the Diamond Princess is warranted. It’s probably warranted, but depending on how strongly cruise ship passengers are pre-selected against having unusually bad health, and depending on how strongly pre-existing health conditions affect someone’s survival prospect for Covid-19, it’s conceivable that the 1% figure doesn’t need to be downward adjusted at all.
To conclude, I don’t understand how age adjustments for SARS-Cov-2 infections on the Diamond Princess can drive down the estimated IFR substantially below 0.5%. 0.5% seems closer to a lower bound to me than anything else. (Of course, those are point estimates. I don’t have strong views on whether 0.125% is outside some appropriate confidence interval, but my impression was that 0.125% was Ioannidis’s point estimate, and interpreted as such, it seems clearly much too low!)
UPDATE: Oh I see. I found an age table that I had overlooked initially. It turns out cruises are really popular for people at age 70-79 (there are about 20% more people of that age than 60-69, whereas it’s the other way around for US demographics). This distribution makes Ioannidis’s figures look more plausible, though the difference doesn’t seem large enough to fully bridge the gap between 0.3% and 0.125%, especially because the 80-89 bracket seems to be represented proportionally again. Still, I don’t anymore think that 0.125% is horribly off.