I’ve been playing with the Kinsa Health weathermap data to get a sense of how effective US lockdowns have been at reducing US fever. The main thing I am interested in is the question of whether lockdown has reduced coronavirus’s r0 below 1 (stopping the spread) or not (reducing spread-rate but not stopping it). I’ve seen evidence that Spain’s complete lockdown has not worked so my expectation is that this is probably the case here. Also, Kinsa’s data has two important caveats:
People who own smart thermometers are more likely to be health conscious which makes them more likely to be health conscious than the overall population. Kinsa may therefore overstate the effect of the lockdown by not effectively sampling the health apathetic people more likely to get the virus.
Kinsa data cannot separate coronavirus fever symptoms with flu fever symptoms. At the early stages of coronavirus spread, seasonal flu illness dominates coronavirus illness and seasonal flu r0 is between 1-2. This means that a lockdown can easily eliminate symptoms caused by seasonal flu illness by reducing flu r0 below zero without reducing coronavirus’s r0 below zero.
I’m addressing this by comparing the largest amounts of observed atypical illness over the last month in different locations with their current total illness to get a conservative estimate of how much coronavirus %ill have changed.
With this in mind, my overall conclusion is that the Kinsa data does not disconfirm the possibility that we’ve reduced r0 below 1. Within the population of people who use smart thermometer’s, we’ve probably stopped the spread but it may/may not have stopped in the overall population. Here are my specific observations:
The overall US %ill weakly suggests we may have reduced r0 below 1. It maxed out at around 5.1% ill compared to a range of 3.7-4.7 %ill . This indicates that 0.4-1.4% of overall illness was due to coronavirus and currently total illness is only 0.88%. This means that, for many values in that range, our lockdowns are actually cutting into the percent of people getting coronavirus and therefore that the virus is not growing.
New York county NY %ill weakly suggests that we may have reduced r0 below 1. It maxed out at 6.4 %ill compared to a typical range of 2.75-4.32, indicating that 2.1-3.65% of people had coronavirus. Currently, total illness is 2.56%. Again, for most values in that range, it looks like we’re reducing the absolute amount of coronavirus.
Cook county IL (Chicago) %ill is very weakly positive on reducing r0 below 1. It maxed out at 5.4 %ill with a range of 2.8-4.9 indicating that 0.5-2.6% of people had coronavirus. Currently the total is 0.92% which suggests we’ve likely cut into coronavirus illness. The range of typical values is so large though that its hard to reach a conclusion
Essex country NJ (Newark) %ill doesn’t say much about r0. It maxed out at 6.1 compared to a typical range of 2.9-4.5 which implies a range of coronavirus %ill of 1.6-3.2 The current value is 2.63% which is closer to the higher end of the range so there’s no evidence that we’ve reduced the amount of coronavirus. Still %ill is continuing to trend down so this may change in the future.
On Mar28, the overall US %ill changed from a steep linear drop of ~-0.3%ill/day to a weaker linear drop of ~-0.1%ill/day. Also, on Mar28, both Newark’s and New York’s fast linear drop is broken with a slight increase in illness and it looks like we’re on our second leg down there now. Similar on Mar27, Chicago’s fast linear drop is broken with a a brief plateau and second leg down. No idea why this happened.
The Kinsa data is barely even weak evidence in favor of R0 < 1. The downward trend in fever readings are confounded, likely severely, by their thermometers having to be actively used vs. being a passive wearable. It seems plausible that more people will check their temperature when they are concerned about COVID-19, and since most people are healthy this will spuriously drive average fever readings down. Plausibly the timing of increased thermometer use will coincide somewhat with shelter-in-place orders since they correlate with severity & awareness of the local outbreak.
Their FAQ notes that they have seen 2-3x normal usage of their thermometers (this was as of March 29, they haven’t updated this part of their FAQ since) and consider this “healthcare seeking behavior” a potential driver of their trends. This has not stopped them from promoting their data to government agencies and NYT, without mentioning this or any other limitations whatsoever (at least to the NYT).
I was completely wrong, I don’t think their data is subject to this worry. They now have a preprint up. From supplementary methods:
We define daily fever counts as the number of unique users per region that take multiple elevated temperature (37.7 C) readings over the past week, and then normalize these counts by the estimated number of unique users who have used the thermometer over the past year.
So lots of repeat readings shouldn’t affect the gauge, and neither should more of their user base taking readings. Unless they are seeing a lot of new users, or lots of returning users that haven’t used the thermometer in over a year, both of which seem somewhat unlikely, their metric should be fine.
Thanks for pointing this out. Having recently looked at Ohio County KY, I think this is correct. %ill there max’d out at above 1% the typical range but has since dropped below 0.4% of the typical range and started rising again (which is notable in contrast with seasonal trends) [Edit to point out that this is true for many counties in the Kentucky/Tennessee area]. This basically demonstrates that having a reported %ill now that is lower than previous in the Kinsa database is insufficient to show r0<1. Probably best to stick with the prior of containment failure.
I’ve been playing with the Kinsa Health weathermap data to get a sense of how effective US lockdowns have been at reducing US fever. The main thing I am interested in is the question of whether lockdown has reduced coronavirus’s r0 below 1 (stopping the spread) or not (reducing spread-rate but not stopping it). I’ve seen evidence that Spain’s complete lockdown has not worked so my expectation is that this is probably the case here. Also, Kinsa’s data has two important caveats:
People who own smart thermometers are more likely to be health conscious which makes them more likely to be health conscious than the overall population. Kinsa may therefore overstate the effect of the lockdown by not effectively sampling the health apathetic people more likely to get the virus.
Kinsa data cannot separate coronavirus fever symptoms with flu fever symptoms. At the early stages of coronavirus spread, seasonal flu illness dominates coronavirus illness and seasonal flu r0 is between 1-2. This means that a lockdown can easily eliminate symptoms caused by seasonal flu illness by reducing flu r0 below zero without reducing coronavirus’s r0 below zero.
I’m addressing this by comparing the largest amounts of observed atypical illness over the last month in different locations with their current total illness to get a conservative estimate of how much coronavirus %ill have changed.
With this in mind, my overall conclusion is that the Kinsa data does not disconfirm the possibility that we’ve reduced r0 below 1. Within the population of people who use smart thermometer’s, we’ve probably stopped the spread but it may/may not have stopped in the overall population. Here are my specific observations:
The overall US %ill weakly suggests we may have reduced r0 below 1. It maxed out at around 5.1% ill compared to a range of 3.7-4.7 %ill . This indicates that 0.4-1.4% of overall illness was due to coronavirus and currently total illness is only 0.88%. This means that, for many values in that range, our lockdowns are actually cutting into the percent of people getting coronavirus and therefore that the virus is not growing.
New York county NY %ill weakly suggests that we may have reduced r0 below 1. It maxed out at 6.4 %ill compared to a typical range of 2.75-4.32, indicating that 2.1-3.65% of people had coronavirus. Currently, total illness is 2.56%. Again, for most values in that range, it looks like we’re reducing the absolute amount of coronavirus.
Cook county IL (Chicago) %ill is very weakly positive on reducing r0 below 1. It maxed out at 5.4 %ill with a range of 2.8-4.9 indicating that 0.5-2.6% of people had coronavirus. Currently the total is 0.92% which suggests we’ve likely cut into coronavirus illness. The range of typical values is so large though that its hard to reach a conclusion
Essex country NJ (Newark) %ill doesn’t say much about r0. It maxed out at 6.1 compared to a typical range of 2.9-4.5 which implies a range of coronavirus %ill of 1.6-3.2 The current value is 2.63% which is closer to the higher end of the range so there’s no evidence that we’ve reduced the amount of coronavirus. Still %ill is continuing to trend down so this may change in the future.
I also considered looking at Santa Clara County CA, Los Angeles County CA, and Orleans Parish LA (New Orleans) but their %ill never exceeded the atypical value by a large enough amount for me to perform comparison.
On Mar28, the overall US %ill changed from a steep linear drop of ~-0.3%ill/day to a weaker linear drop of ~-0.1%ill/day. Also, on Mar28, both Newark’s and New York’s fast linear drop is broken with a slight increase in illness and it looks like we’re on our second leg down there now. Similar on Mar27, Chicago’s fast linear drop is broken with a a brief plateau and second leg down. No idea why this happened.
The Kinsa data is barely even weak evidence in favor of R0 < 1. The downward trend in fever readings are confounded, likely severely, by their thermometers having to be actively used vs. being a passive wearable. It seems plausible that more people will check their temperature when they are concerned about COVID-19, and since most people are healthy this will spuriously drive average fever readings down. Plausibly the timing of increased thermometer use will coincide somewhat with shelter-in-place orders since they correlate with severity & awareness of the local outbreak.
Their FAQ notes that they have seen 2-3x normal usage of their thermometers (this was as of March 29, they haven’t updated this part of their FAQ since) and consider this “healthcare seeking behavior” a potential driver of their trends. This has not stopped them from promoting their data to government agencies and NYT, without mentioning this or any other limitations whatsoever (at least to the NYT).
I was completely wrong, I don’t think their data is subject to this worry. They now have a preprint up. From supplementary methods:
So lots of repeat readings shouldn’t affect the gauge, and neither should more of their user base taking readings. Unless they are seeing a lot of new users, or lots of returning users that haven’t used the thermometer in over a year, both of which seem somewhat unlikely, their metric should be fine.
Thanks for pointing this out. Having recently looked at Ohio County KY, I think this is correct. %ill there max’d out at above 1% the typical range but has since dropped below 0.4% of the typical range and started rising again (which is notable in contrast with seasonal trends) [Edit to point out that this is true for many counties in the Kentucky/Tennessee area]. This basically demonstrates that having a reported %ill now that is lower than previous in the Kinsa database is insufficient to show r0<1. Probably best to stick with the prior of containment failure.